________________________________________ Från: users-bounces@ovirt.org [users-bounces@ovirt.org] för Dan Kenigsberg [danken@redhat.com] Skickat: den 9 januari 2014 17:33 Till: Sven Kieske Kopia: users@ovirt.org Ämne: Re: [Users] VSDM´s logrotate makes Hosts fill up var eventually On Thu, Jan 09, 2014 at 01:39:08PM +0000, Sven Kieske wrote:
Hi,
I also guess this gets so large because of the loglevels in /etc/vdsm/logger.conf
this seems to be the default:
[logger_root] level=DEBUG handlers=syslog,logfile propagate=0
[logger_vds] level=DEBUG handlers=syslog,logfile qualname=vds propagate=0
[logger_Storage] level=DEBUG handlers=logfile qualname=Storage propagate=0
[logger_metadata] level=WARNING handlers=metadata qualname=irs.metadata propagate=0
[handler_syslog] level=WARNING class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER)
[handler_logfile] class=logging.handlers.WatchedFileHandler args=('/var/log/vdsm/vdsm.log',) filters=storage.misc.TracebackRepeatFilter level=DEBUG formatter=long
[handler_metadata] class=logging.handlers.WatchedFileHandler args=('/var/log/vdsm/metadata.log',) level=WARNING formatter=long
which is "debug" level for most loggers.
Question to the devs:
Is this really needed as a default in a production environment?
my vdsm is a little bit older btw:
vdsm-4.12.1-4.el6.x86_64 vdsm-cli-4.12.1-4.el6.noarch vdsm-python-4.12.1-4.el6.x86_64 vdsm-python-cpopen-4.12.1-4.el6.x86_64 vdsm-xmlrpc-4.12.1-4.el6.noarch
did this change in vdsm 4.13. ?
No change yet.
Am 09.01.2014 14:26, schrieb Karli Sjöberg:
Hi!
I just noticed my Hypervisor nodes starting to complain about disks almost being full. I started investigation and noticed that: # du -h /var/log/libvirtd.log 100G /var/log/libvirtd.log
And many Hosts system partition had indeed become full:S
Why weren´t the file rotated? Well: # ls -lah /var/log/libvirtd.log.* | wc -l 100
And the rotate policy says: /etc/logrotate.d/libvirtd ## beginning of configuration section by vdsm /var/log/libvirt/libvirtd.log { rotate 100 missingok copytruncate size 15M compress compresscmd /usr/bin/xz uncompresscmd /usr/bin/unxz compressext .xz }
Now, I just handled it by changing "100" to "1000" but I think that a
I do not understand this issue, Karli. After 100 log files have been created, the oldest one should have been removed and replaced by the newest one. logrotate is expected to be called every 15 minutes, so it should not have stayed above 15M for so long. Do you see any error when running `/usr/sbin/logrotate /etc/logrotate.d/libvirtd` as root? What the funk, it doesn´t do anything, complains nothing like it´s done what it´s supposed to, and no errors in "/var/log/messages" either...
better default needs to be placed in general, don´t you? # rpm -qa | grep vdsm vdsm-python-4.13.0-11.el6.x86_64 vdsm-python-cpopen-4.13.0-11.el6.x86_64 vdsm-4.13.0-11.el6.x86_64 vdsm-xmlrpc-4.13.0-11.el6.noarch vdsm-cli-4.13.0-11.el6.noarch
The question of how much logging we should keep is a tough one. I, as a developer, would like to have as much as possible. For long-running busy systems, it has happened to me that the core bug was spotted in vdsm.log.67 or so. However, I understand that verbosity has its price. To understand whether we are stable enough to change the defaults, I need volunteers: people who are willing to change their log level to INFO or WARNING, and see if they miss useful information from their logs. When you make you log level higher, you can lower the number of kept log files, as they would not be filled as quick. Would you, users@, help me with hard data? Well, I understand that having more data of course helps when there´s an issue, however having lots of data isn´t helping in this particular case;P Seriously though, it´s not bothering me having lots of logs. I don´t mind the DEBUG mode, and as long as rotate _does what it´s supposed to_ (but that´s besides the point) it´s not taking up that much space to argue about either so for me it´s cool (normally). Dan. _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users