On Thu, Mar 3, 2016 at 4:28 PM, Martin Polednik <mpolednik@redhat.com> wrote:

Hello developers!

I've been chasing a bug that lead me to an idea of improving our XML
logging. Now, to see a VMs generated libvirt XML, we have to rely on
vdsm.log. The issue is that the log is rotating and therefore, it is
easy to miss the correct XML when dealing with busy hypervisor.

Since we're built on libvirt, I was thinking of doing similar thinks
that libvirt does with qemu commandline. Each running domain(VM) has
it's command line logged in /var/log/libvirt/qemu/${vmname}. This is
great for debugging as you can mostly just take the cmdline and
restart the VM.

There is an issue with using the cmdline directly - networking.
Libvirt uses additional script to create and up a bridge. Therefore,
it is easier to use the XML and shape it to one's needs.

I propose that we properly log the generated XML in a similar fashion
as libvirt generates the cmdline. This means we would have path like
/var/log/vdsm/libvirt/${vmname}, where generated XML would be stored.
To minimize the logging requirements, only last definition of VM with
that name may be stored. Additionally, exception level errors
related to that VM could also be stored there.

Without history, it will much less useful, so I think we should keep this in vdsm log.

How about creating separate log for virt?

We talked about separating the application to several parts, so we can start by

having separate logs. In /var/log/vdsm/virt.log (or /var/log/vdsm/vms.log) you

will see calls to the storage layer (service in the future), but you will not see all

the noise that storage generates (e.g. resourseManager crap).

What do you think, can we afford the space and additional writes per
VM to help the debugging process?

We can, but the default log level should not show stuff like full xml dumps.

Nir