oVirt log analyzer
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors. The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-) You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it. Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements. Regards, Milan
Nice! I think a nice RFE would be to surface this info in the UI. On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 29 March 2018 at 21:11, Tomas Jelinek <tjelinek@redhat.com> wrote:
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
I wonder if we can automate running it on OST failures to get automated failure analysis.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
Barak Korren <bkorren@redhat.com> writes:
On 29 March 2018 at 21:11, Tomas Jelinek <tjelinek@redhat.com> wrote:
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
I wonder if we can automate running it on OST failures to get automated failure analysis.
Well, ovirt-log-analyzer doesn't provide failure analysis, it just tries to extract more important lines from the logs and add some information to them. It may or may not be useful to add its output to OST failures in future, but some feedback and testing are needed first, I assume there are still bugs and things to improve.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 3 April 2018 at 12:55, Milan Zamazal <mzamazal@redhat.com> wrote:
Barak Korren <bkorren@redhat.com> writes:
On 29 March 2018 at 21:11, Tomas Jelinek <tjelinek@redhat.com> wrote:
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
I wonder if we can automate running it on OST failures to get automated failure analysis.
Well, ovirt-log-analyzer doesn't provide failure analysis, it just tries to extract more important lines from the logs and add some information to them. It may or may not be useful to add its output to OST failures in future, but some feedback and testing are needed first, I assume there are still bugs and things to improve.
I'll try to be more specific about the use case we're lookin at. Right now, when a test fails in OST - al you get is the traceback from nose of the oVirt API cll you've made. For some tests additional information is provided by having nise collect it from STDOUT. It could be very useful if we could use some information about the API call that had been made, and use it to extract relevant information about the call from the vdms and engine logs, Can ovirt-log-analyzer be used for that? -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
Barak Korren <bkorren@redhat.com> writes:
Right now, when a test fails in OST - al you get is the traceback from nose of the oVirt API cll you've made. For some tests additional information is provided by having nise collect it from STDOUT.
It could be very useful if we could use some information about the API call that had been made, and use it to extract relevant information about the call from the vdms and engine logs,
Can ovirt-log-analyzer be used for that?
Maybe, but not out of the box. It can parse logs and find some relations between Engine and Vdsm logs, so it might be extended to do that. However it's more focused on guess work. Depending on kind of the information to be provided there could be simpler ways to get it.
On Wed, Apr 4, 2018 at 10:47 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Barak Korren <bkorren@redhat.com> writes:
Right now, when a test fails in OST - al you get is the traceback from nose of the oVirt API cll you've made. For some tests additional information is provided by having nise collect it from STDOUT.
It could be very useful if we could use some information about the API call that had been made, and use it to extract relevant information about the call from the vdms and engine logs,
Can ovirt-log-analyzer be used for that?
Maybe, but not out of the box. It can parse logs and find some relations between Engine and Vdsm logs, so it might be extended to do that. However it's more focused on guess work. Depending on kind of the information to be provided there could be simpler ways to get it.
Is correlation id not enough to find the relations?
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Piotr Kliczewski <piotr.kliczewski@gmail.com> writes:
On Wed, Apr 4, 2018 at 10:47 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Barak Korren <bkorren@redhat.com> writes:
Right now, when a test fails in OST - al you get is the traceback from nose of the oVirt API cll you've made. For some tests additional information is provided by having nise collect it from STDOUT.
It could be very useful if we could use some information about the API call that had been made, and use it to extract relevant information about the call from the vdms and engine logs,
Can ovirt-log-analyzer be used for that?
Maybe, but not out of the box. It can parse logs and find some relations between Engine and Vdsm logs, so it might be extended to do that. However it's more focused on guess work. Depending on kind of the information to be provided there could be simpler ways to get it.
Is correlation id not enough to find the relations?
It is enough to find relations between API calls and something like grep may be easier to use for that than ovirt-log-analyzer. Again, it depends on what kind of information should be provided.
On Thu, Mar 29, 2018 at 9:11 PM, Tomas Jelinek <tjelinek@redhat.com> wrote:
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
This is not very clear from the readme, which only says 'Assuming your *oVirt logs* are stored in DIRECTORY ' - what logs exactly are in that directory? Is that the result of logs from ovirt-log-collector ? Y.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Yaniv Kaul <ykaul@redhat.com> writes:
On Thu, Mar 29, 2018 at 9:11 PM, Tomas Jelinek <tjelinek@redhat.com> wrote:
On Thu, Mar 29, 2018 at 7:55 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Nice! I think a nice RFE would be to surface this info in the UI.
On Thu, Mar 29, 2018 at 8:30 AM, Milan Zamazal <mzamazal@redhat.com> wrote:
Hi, during last year Outreachy internship a tool for analyzing oVirt logs was created. When it is provided with oVirt logs (such as SOS reports, logs gathered by Lago, single or multiple log files) it tries to identify and classify important lines from the logs and present them in a structured form. Its primary purpose is to get a quick and easy overview of actions and errors.
I would add that it can correlate more log files (from engine/vdsm/libvirt/quemu) and show a unified view of them. It can follow the life of one entity (such as a VM) and show what was going on with it across the system. I have used it a lot to look for races and it was pretty useful for that.
This is not very clear from the readme, which only says 'Assuming your *oVirt logs* are stored in DIRECTORY ' - what logs exactly are in that directory?
Whatever logs you have got, unpacked (i.e. not in tar), either uncompressed or compressed with xz or gzip. The analyzer looks for Engine, Vdsm, libvirt, QEMU, sanlock, and SPM lock logs by default, identified by file names. Directory structure doesn't matter.
Is that the result of logs from ovirt-log-collector ? Y.
The tool analyses given logs and produces text files with the extracted information. There is an Emacs user interface that presents the output in a nice way with added functionality such as filtering. Emacs haters can use the plain text files or write another user interface. :-)
You can get ovirt-log-analyzer from https://github.com/mz-pdm/ovirt-log-analyzer README.md explains how to use it.
Note that ovirt-log-analyzer has been created within the limited resources of an Outreachy internship with some additional work and not everything is perfect. Feel free to make improvements.
Regards, Milan _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (6)
-
Barak Korren -
Greg Sheremeta -
Milan Zamazal -
Piotr Kliczewski -
Tomas Jelinek -
Yaniv Kaul