oVirt hosted engine agent and broker duplicate logs to syslog
Hi, After upgrade from 3.6.0 to 3.6.1 agent and broker duplicate their logs to syslog. So, the same messages logged twice to files in /var/log/ovirt-hosted-engine-ha/ directory and to /var/log/messages file. Agent and broker configuration files remain the same for 3.5, 3.6.0 and 3.6.1 and there is not such logs duplication in 3.5 and 3.6.0. Is it a bug or expected behavior? OS is CentOS 7.2 # rpm -qa 'ovirt*' ovirt-vmconsole-1.0.0-1.el7.centos.noarch ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch ovirt-hosted-engine-ha-1.3.3.5-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-release36-002-2.noarch ovirt-setup-lib-1.0.0-1.el7.centos.noarch ovirt-hosted-engine-setup-1.3.1.3-1.el7.centos.noarch # cat /etc/ovirt-hosted-engine-ha/agent-log.conf [loggers] keys=root [handlers] keys=syslog,logfile [formatters] keys=long,sysform [logger_root] level=INFO handlers=syslog,logfile propagate=0 [handler_syslog] level=ERROR class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER) [handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/agent.log', 'd', 1, 7) level=DEBUG formatter=long [formatter_long] format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s [formatter_sysform] format=ovirt-ha-agent %(name)s %(levelname)s %(message)s datefmt= Aleksey
Il 31/Dic/2015 16:36, "Aleksey Chudov" <aleksey.chudov@gmail.com> ha scritto:
Hi,
After upgrade from 3.6.0 to 3.6.1 agent and broker duplicate their logs
to syslog. So, the same messages logged twice to files in /var/log/ovirt-hosted-engine-ha/ directory and to /var/log/messages file.
Agent and broker configuration files remain the same for 3.5, 3.6.0 and
3.6.1 and there is not such logs duplication in 3.5 and 3.6.0.
Is it a bug or expected behavior?
I think that it's an expected behavior: for 3.6.1 we had to rewrite the systemd startup script due to a slightly different behavior on centos 7.2. Prior than that the agent was forking as a daemon while now it's running as a systemd service with type=simple so its output goes to systemd logging facilities and so also in /var/log/messages according to your system configuration.
OS is CentOS 7.2
# rpm -qa 'ovirt*' ovirt-vmconsole-1.0.0-1.el7.centos.noarch ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch ovirt-hosted-engine-ha-1.3.3.5-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-release36-002-2.noarch ovirt-setup-lib-1.0.0-1.el7.centos.noarch ovirt-hosted-engine-setup-1.3.1.3-1.el7.centos.noarch
# cat /etc/ovirt-hosted-engine-ha/agent-log.conf [loggers] keys=root
[handlers] keys=syslog,logfile
[formatters] keys=long,sysform
[logger_root] level=INFO handlers=syslog,logfile propagate=0
[handler_syslog] level=ERROR class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER)
[handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/agent.log', 'd', 1, 7) level=DEBUG formatter=long
[formatter_long]
format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s
[formatter_sysform] format=ovirt-ha-agent %(name)s %(levelname)s %(message)s datefmt=
Aleksey
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Thank you for the answer. So, currently there is at least three copy of agent and broker logs on centos 7.2: 1. agent.log and broker.log files in /var/log/ovirt-hosted-engine-ha/ directory 2. /var/log/messages file 3. journald database Does errors additionally send to syslog according to agent-log.conf and broker-log.conf files in /etc/ovirt-hosted-engine-ha/ directory? It's a bit too much :) Do you plan to fix this duplication? For example 2. and 3. can be quickly disabled by adding StandardOutput=null option to unit file [Service] ... StandardOutput=null On Sat, Jan 2, 2016 at 2:01 PM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Il 31/Dic/2015 16:36, "Aleksey Chudov" <aleksey.chudov@gmail.com> ha scritto:
Hi,
After upgrade from 3.6.0 to 3.6.1 agent and broker duplicate their logs
to syslog. So, the same messages logged twice to files in /var/log/ovirt-hosted-engine-ha/ directory and to /var/log/messages file.
Agent and broker configuration files remain the same for 3.5, 3.6.0 and
3.6.1 and there is not such logs duplication in 3.5 and 3.6.0.
Is it a bug or expected behavior?
I think that it's an expected behavior: for 3.6.1 we had to rewrite the systemd startup script due to a slightly different behavior on centos 7.2. Prior than that the agent was forking as a daemon while now it's running as a systemd service with type=simple so its output goes to systemd logging facilities and so also in /var/log/messages according to your system configuration.
OS is CentOS 7.2
# rpm -qa 'ovirt*' ovirt-vmconsole-1.0.0-1.el7.centos.noarch ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch ovirt-hosted-engine-ha-1.3.3.5-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-release36-002-2.noarch ovirt-setup-lib-1.0.0-1.el7.centos.noarch ovirt-hosted-engine-setup-1.3.1.3-1.el7.centos.noarch
# cat /etc/ovirt-hosted-engine-ha/agent-log.conf [loggers] keys=root
[handlers] keys=syslog,logfile
[formatters] keys=long,sysform
[logger_root] level=INFO handlers=syslog,logfile propagate=0
[handler_syslog] level=ERROR class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER)
[handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/agent.log', 'd', 1, 7) level=DEBUG formatter=long
[formatter_long]
format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s
[formatter_sysform] format=ovirt-ha-agent %(name)s %(levelname)s %(message)s datefmt=
Aleksey
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Il 02/Gen/2016 13:29, "Aleksey Chudov" <aleksey.chudov@gmail.com> ha scritto:
Thank you for the answer.
So, currently there is at least three copy of agent and broker logs on
1. agent.log and broker.log files in /var/log/ovirt-hosted-engine-ha/
centos 7.2: directory
2. /var/log/messages file 3. journald database
Does errors additionally send to syslog according to agent-log.conf and broker-log.conf files in /etc/ovirt-hosted-engine-ha/ directory?
It's a bit too much :) Do you plan to fix this duplication?
For example 2. and 3. can be quickly disabled by adding StandardOutput=null option to unit file
[Service] ... StandardOutput=null
On Sat, Jan 2, 2016 at 2:01 PM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Il 31/Dic/2015 16:36, "Aleksey Chudov" <aleksey.chudov@gmail.com> ha
scritto:
Hi,
After upgrade from 3.6.0 to 3.6.1 agent and broker duplicate their
logs to syslog. So, the same messages logged twice to files in /var/log/ovirt-hosted-engine-ha/ directory and to /var/log/messages file.
Agent and broker configuration files remain the same for 3.5, 3.6.0
and 3.6.1 and there is not such logs duplication in 3.5 and 3.6.0.
Is it a bug or expected behavior?
I think that it's an expected behavior: for 3.6.1 we had to rewrite the systemd startup script due to a slightly different behavior on centos 7.2. Prior than that the agent was forking as a daemon while now it's running as a systemd service with type=simple so its output goes to systemd logging facilities and so also in /var/log/messages according to your system configuration.
OS is CentOS 7.2
# rpm -qa 'ovirt*' ovirt-vmconsole-1.0.0-1.el7.centos.noarch ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch ovirt-hosted-engine-ha-1.3.3.5-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-release36-002-2.noarch ovirt-setup-lib-1.0.0-1.el7.centos.noarch ovirt-hosted-engine-setup-1.3.1.3-1.el7.centos.noarch
# cat /etc/ovirt-hosted-engine-ha/agent-log.conf [loggers] keys=root
[handlers] keys=syslog,logfile
[formatters] keys=long,sysform
[logger_root] level=INFO handlers=syslog,logfile propagate=0
[handler_syslog] level=ERROR class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER)
[handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/agent.log', 'd', 1, 7) level=DEBUG formatter=long
[formatter_long]
If it's too much, I'd prefer to skip the dedicated log file one. Personally I'm quite comfortable with systemd logging. format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s
[formatter_sysform] format=ovirt-ha-agent %(name)s %(levelname)s %(message)s datefmt=
Aleksey
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Sat, Jan 2, 2016 at 2:29 PM, Aleksey Chudov <aleksey.chudov@gmail.com> wrote:
Thank you for the answer.
So, currently there is at least three copy of agent and broker logs on centos 7.2: 1. agent.log and broker.log files in /var/log/ovirt-hosted-engine-ha/ directory 2. /var/log/messages file 3. journald database
I don't think we can fix the duplication in /var/log/messages and journald database. If we log to syslog, it will appear in both of them: # logger `uuidgen` # tail -n 1 /var/log/messages Jan 2 17:27:37 xxx root: b364232b-20d1-4e50-aa71-10a9b4513456 # journalctl -n 1 -- Logs begin at Fri 2015-12-11 22:47:26 IST, end at Sat 2016-01-02 17:27:37 IST. -- Jan 02 17:27:37 xxx root[6828]: b364232b-20d1-4e50-aa71-10a9b4513456
Does errors additionally send to syslog according to agent-log.conf and broker-log.conf files in /etc/ovirt-hosted-engine-ha/ directory?
It's a bit too much :) Do you plan to fix this duplication?
For example 2. and 3. can be quickly disabled by adding StandardOutput=null option to unit file
[Service] ... StandardOutput=null
Did you try that?
On Sat, Jan 2, 2016 at 2:01 PM, Simone Tiraboschi <stirabos@redhat.com> wrote:
Il 31/Dic/2015 16:36, "Aleksey Chudov" <aleksey.chudov@gmail.com> ha scritto:
Hi,
After upgrade from 3.6.0 to 3.6.1 agent and broker duplicate their logs to syslog. So, the same messages logged twice to files in /var/log/ovirt-hosted-engine-ha/ directory and to /var/log/messages file.
Agent and broker configuration files remain the same for 3.5, 3.6.0 and 3.6.1 and there is not such logs duplication in 3.5 and 3.6.0.
Is it a bug or expected behavior?
I think that it's an expected behavior: for 3.6.1 we had to rewrite the systemd startup script due to a slightly different behavior on centos 7.2. Prior than that the agent was forking as a daemon while now it's running as a systemd service with type=simple so its output goes to systemd logging facilities and so also in /var/log/messages according to your system configuration.
How is this related? According to the logger configuration, logs are logged to the log file and to /dev/log, so nothing should be seen in stderr or stdout.
OS is CentOS 7.2
# rpm -qa 'ovirt*' ovirt-vmconsole-1.0.0-1.el7.centos.noarch ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch ovirt-host-deploy-1.4.1-1.el7.centos.noarch ovirt-hosted-engine-ha-1.3.3.5-1.el7.centos.noarch ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch ovirt-release36-002-2.noarch ovirt-setup-lib-1.0.0-1.el7.centos.noarch ovirt-hosted-engine-setup-1.3.1.3-1.el7.centos.noarch
# cat /etc/ovirt-hosted-engine-ha/agent-log.conf [loggers] keys=root
[handlers] keys=syslog,logfile
[formatters] keys=long,sysform
[logger_root] level=INFO handlers=syslog,logfile propagate=0
[handler_syslog] level=ERROR
Only errors should appear in /var/log/messages - do you see other log level there?
class=handlers.SysLogHandler formatter=sysform args=('/dev/log', handlers.SysLogHandler.LOG_USER)
[handler_logfile] class=logging.handlers.TimedRotatingFileHandler args=('/var/log/ovirt-hosted-engine-ha/agent.log', 'd', 1, 7) level=DEBUG formatter=long
[formatter_long]
format=%(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s
[formatter_sysform] format=ovirt-ha-agent %(name)s %(levelname)s %(message)s datefmt=
Aleksey
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (3)
-
Aleksey Chudov -
Nir Soffer -
Simone Tiraboschi