[ovirt-users] vdsm using 100% CPU, rapidly filling logs with _handle_event messages

[Dan Kenigsberg]

On Thu, Nov 12, 2015 at 08:45:43AM -0500, Robert Story wrote:
> I'm running oVirt 3.5.x with a hosted engine. This morning I noticed that 2
> of my 5 hosts were showing 99-100% cpu usage. Logging in to them, vdsmd
> seemed to be the culprit, and it was filling the log file with these
> messages:

You're probably seeing

    Bug 1226911 - vmchannel thread consumes 100% of CPU

which was closed due to missing information. Do you have any information
on when this pops up? Is it reproducible? Would you be bale to test a
suggested patch

    https://gerrit.ovirt.org/#/c/42570/

Regards,
Dan.

> 
> VM Channels Listener::DEBUG::2015-11-12
> 08:09:26,292::vmchannels::59::vds::(_handle_event) Received 00000011. On fd removed by epoll. VM Channels Listener::INFO::2015-11-12 08:09:26,293::vmchannels::54::vds::(_handle_event) Received 00000011 on fileno 119
> VM Channels Listener::DEBUG::2015-11-12 08:09:26,293::vmchannels::59::vds::(_handle_event) Received 00000011. On fd removed by epoll.
> VM Channels Listener::INFO::2015-11-12 08:09:26,293::vmchannels::54::vds::(_handle_event) Received 00000011 on fileno 75
> VM Channels Listener::DEBUG::2015-11-12 08:09:26,293::vmchannels::59::vds::(_handle_event) Received 00000011. On fd removed by epoll.
> VM Channels Listener::INFO::2015-11-12 08:09:26,294::vmchannels::54::vds::(_handle_event) Received 00000011 on fileno 119
> VM Channels Listener::DEBUG::2015-11-12 08:09:26,294::vmchannels::59::vds::(_handle_event) Received 00000011. On fd removed by epoll.
> 
> I googled to see how to change the debug level to turn of DEBUG messages
> for vdsm, which referred me to libvirtd.conf, but the debug level there was
> not set, which should have meant a log level of 3 (warnings and errors), so
> I'm not sure why the log was filling up with DEBUG/INFO messages.
> 
> I restarted vdsmd, which resulted in those nodes being marked as
> 'disconnected', but they did eventually recover and loads went back to
> normal.
> 
> This may or may not be related to the fact that the 3 hosts where this did
> not happen can't seem to keep their ha brokers up. I'll be starting a new
> thread on that shortly.
> 
> 
> Robert
> 
> -- 
> Senior Software Engineer @ Parsons



> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users