Thanks for the quick respons, but there is no memcached running on that VM. Kind regards 2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote:
Dear all,
When we monitor one of our machines, we noticed that there was one vm that was constantly giving a error of memory usage. But when we took a look at it, there is actually nothing wrong with it. Now we looked furhter then that. We looked at the API of the machine and noticed something very strange:
<statistic
href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory used (agent)</description><values type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit>
It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
What could be the problem?
Kind regards,
koen
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
In The GUI, it says it's using 25% of the memory. 2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote:
Dear all,
When we monitor one of our machines, we noticed that there was one vm that was constantly giving a error of memory usage. But when we took a look at it, there is actually nothing wrong with it. Now we looked furhter then that. We looked at the API of the machine and noticed something very strange:
<statistic
href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory used (agent)</description><values type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit>
It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
What could be the problem?
Kind regards,
koen
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote:
In The GUI, it says it's using 25% of the memory.
I guess it's the real value, right? The same happened for the memcached vm, someone reported to me - negative value in REST-API, but correct graph in oVirt webadmin GUI. I fear I have no idea how this can happen - so maybe someone else can help you troubleshoot this issue.
2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote: > Dear all, > > > When we monitor one of our machines, we noticed that there was one vm > that was constantly giving a error of memory usage. But when we took a > look at it, there is actually nothing wrong with it. Now we looked > furhter then that. We looked at the API of the machine and noticed > something very strange: > > <statistic > href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory used (agent)</description><values type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit> > > > It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
> > > What could be the problem? > > > Kind regards, > > koen >
> _______________________________________________ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users
----- Original Message -----
From: "René Koch" <rkoch@linuxland.at> To: "Koen Vanoppen" <vanoppen.koen@gmail.com> Cc: users@ovirt.org Sent: Wednesday, February 12, 2014 4:18:37 PM Subject: Re: [Users] Memory usage
On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote:
In The GUI, it says it's using 25% of the memory.
I guess it's the real value, right? The same happened for the memcached vm, someone reported to me - negative value in REST-API, but correct graph in oVirt webadmin GUI.
I fear I have no idea how this can happen - so maybe someone else can help you troubleshoot this issue.
2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote: > Dear all, > > > When we monitor one of our machines, we noticed that there was one vm > that was constantly giving a error of memory usage. But when we took a > look at it, there is actually nothing wrong with it. Now we looked > furhter then that. We looked at the API of the machine and noticed > something very strange: > > <statistic > href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory used (agent)</description><values type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit> > > > It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
> > > What could be the problem? > > > Kind regards, > > koen >
Guys, these values are usually a result of overcommitment mechanism usage. For example, if KSM is effective, it will free a lot of memory pages, and total-free-committed becomes negative. This was reported in https://bugzilla.redhat.com/show_bug.cgi?id=977758 and the engine is using memFree reported by vdsm, which is more accurate. The API reports the old version due to backwards compatibility. Doron P.S. René- thanks for helping!
On Wed, 2014-02-12 at 11:22 -0500, Doron Fediuck wrote:
----- Original Message -----
From: "René Koch" <rkoch@linuxland.at> To: "Koen Vanoppen" <vanoppen.koen@gmail.com> Cc: users@ovirt.org Sent: Wednesday, February 12, 2014 4:18:37 PM Subject: Re: [Users] Memory usage
On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote:
In The GUI, it says it's using 25% of the memory.
I guess it's the real value, right? The same happened for the memcached vm, someone reported to me - negative value in REST-API, but correct graph in oVirt webadmin GUI.
I fear I have no idea how this can happen - so maybe someone else can help you troubleshoot this issue.
2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote: > Dear all, > > > When we monitor one of our machines, we noticed that there was one vm > that was constantly giving a error of memory usage. But when we took a > look at it, there is actually nothing wrong with it. Now we looked > furhter then that. We looked at the API of the machine and noticed > something very strange: > > <statistic > href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory used (agent)</description><values type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit> > > > It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
> > > What could be the problem? > > > Kind regards, > > koen >
Guys, these values are usually a result of overcommitment mechanism usage. For example, if KSM is effective, it will free a lot of memory pages, and total-free-committed becomes negative.
Thanks a lot for the information.
This was reported in https://bugzilla.redhat.com/show_bug.cgi?id=977758 and the engine is using memFree reported by vdsm, which is more accurate.
The API reports the old version due to backwards compatibility.
I just had a look at the bugzilla report and the RHEV documentation which says "Current memory in bytes used by the virtual machine.". So this means, the reported values are totally useless for monitoring memory usage of a virtual machine if KSM is active. I would expect to get the memory usage of a virtual machine and not how much memory is consumed on the hypervisor (this is pretty useless information for me). Is it planned to report the memory usage in a virtual machine in the API as well? Regards, René
Doron
P.S. René- thanks for helping!
Hi, I'm interested in the very same question. Additionally I don't see any way how this would break backwards compatibility. In contrary, in 3.4 some other values in the API change in a not backward compatible way, which actually do what they are supposed to do, so even if it would break compatibility this wouldn't clearly be a "no go" as it is done in other parts of the API too. In case you don't know which values I mean: I talk about the JSON implementation. I think the old value is pretty useless and needs to be fixed. So I don't see how a fix to this value should break any compatibility. I would be very interested in seeing the actual code which relies on negative values of consumed ram inside a vm. What would you do with it? Am 12.02.2014 18:31, schrieb René Koch:
I just had a look at the bugzilla report and the RHEV documentation which says "Current memory in bytes used by the virtual machine.". So this means, the reported values are totally useless for monitoring memory usage of a virtual machine if KSM is active. I would expect to get the memory usage of a virtual machine and not how much memory is consumed on the hypervisor (this is pretty useless information for me).
Is it planned to report the memory usage in a virtual machine in the API as well?
-- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
hello rene it's during testing check_rhev3 that we found this issue do you see a workaround for this? On Wed, Feb 12, 2014 at 6:31 PM, René Koch <rkoch@linuxland.at> wrote:
On Wed, 2014-02-12 at 11:22 -0500, Doron Fediuck wrote:
----- Original Message -----
From: "René Koch" <rkoch@linuxland.at> To: "Koen Vanoppen" <vanoppen.koen@gmail.com> Cc: users@ovirt.org Sent: Wednesday, February 12, 2014 4:18:37 PM Subject: Re: [Users] Memory usage
On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote:
In The GUI, it says it's using 25% of the memory.
I guess it's the real value, right? The same happened for the memcached vm, someone reported to me - negative value in REST-API, but correct graph in oVirt webadmin GUI.
I fear I have no idea how this can happen - so maybe someone else can help you troubleshoot this issue.
2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote: > Dear all, > > > When we monitor one of our machines, we noticed
there was one vm > that was constantly giving a error of memory usage. But when we took a > look at it, there is actually nothing wrong with
it.
Now we looked > furhter then that. We looked at the API of the machine and noticed > something very strange: > > <statistic >
that href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08"
id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory
used (agent)</description><values
type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit>
> > > It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
> > > What could be the problem? > > > Kind regards, > > koen >
Guys, these values are usually a result of overcommitment mechanism usage. For example, if KSM is effective, it will free a lot of memory pages, and total-free-committed becomes negative.
Thanks a lot for the information.
This was reported in https://bugzilla.redhat.com/show_bug.cgi?id=977758 and the engine is using memFree reported by vdsm, which is more accurate.
The API reports the old version due to backwards compatibility.
I just had a look at the bugzilla report and the RHEV documentation which says "Current memory in bytes used by the virtual machine.". So this means, the reported values are totally useless for monitoring memory usage of a virtual machine if KSM is active. I would expect to get the memory usage of a virtual machine and not how much memory is consumed on the hypervisor (this is pretty useless information for me).
Is it planned to report the memory usage in a virtual machine in the API as well?
Regards, René
Doron
P.S. René- thanks for helping!
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Thu, 2014-02-13 at 09:39 +0100, david van zeebroeck wrote:
hello rene it's during testing check_rhev3 that we found this issue do you see a workaround for this?
At the moment I can only set the memory usage to 0 and add comments to the documentation that vm memory usage is the usage of the vm on the host, but not the memory usage in the vm (which I thought it was). This issue was reported once to me, and check_rhev3 has a calculation of the memory usage if it's negative in version 1.3, but this only solved the issue accidentally for this specific vm of this user. As Doron explained me in previous mail what this memory usage value really means, I'll revert the commit from https://github.com/ovido/check_rhev3/issues/10 and set the memory usage to 0 if negative. I never saw this issue in my environments or on customer sides, as we normally always monitor vms using agents in the vm and not using check_rhev3 for vms. I use it on some test machines in my environments, but have KSM disabled, as it consumes too much CPU load when having databases running in vms. Workaround is to get the memory usage from polling agents which are running directly in the vm like check_mk, snmp, nrpe,... until I can receive the memory usage in the vm via REST-API and add it to check_rhev3. Regards, René
On Wed, Feb 12, 2014 at 6:31 PM, René Koch <rkoch@linuxland.at> wrote: On Wed, 2014-02-12 at 11:22 -0500, Doron Fediuck wrote: > > ----- Original Message ----- > > From: "René Koch" <rkoch@linuxland.at> > > To: "Koen Vanoppen" <vanoppen.koen@gmail.com> > > Cc: users@ovirt.org > > Sent: Wednesday, February 12, 2014 4:18:37 PM > > Subject: Re: [Users] Memory usage > > > > On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote: > > > In The GUI, it says it's using 25% of the memory. > > > > > > I guess it's the real value, right? > > The same happened for the memcached vm, someone reported to me - > > negative value in REST-API, but correct graph in oVirt webadmin GUI. > > > > I fear I have no idea how this can happen - so maybe someone else can > > help you troubleshoot this issue. > > > > > > > > > > > > 2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: > > > Thanks for the quick respons, but there is no memcached > > > running on that VM. > > > > > > > > > Kind regards > > > > > > > > > > > > 2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>: > > > > > > On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen > > > wrote: > > > > Dear all, > > > > > > > > > > > > When we monitor one of our machines, we noticed that > > > there was one vm > > > > that was constantly giving a error of memory usage. > > > But when we took a > > > > look at it, there is actually nothing wrong with it. > > > Now we looked > > > > furhter then that. We looked at the API of the > > > machine and noticed > > > > something very strange: > > > > > > > > <statistic > > > > > > > href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08" > > > id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory > > > used (agent)</description><values > > > type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit> > > > > > > > > > > > > It's a negative... > > > > > > > > > > > > Do you have memcached running in this vm? > > > > > > I heard about this issue with memcached, but never > > > tested memcached in > > > my oVirt environment. You get the real usage value > > > with > > > memory.used = memory.installed + memory.used > > > > > > > > > Regards, > > > René > > > > > > > > > > > > > > > What could be the problem? > > > > > > > > > > > > Kind regards, > > > > > > > > koen > > > > > > > > > Guys, > these values are usually a result of overcommitment mechanism usage. > For example, if KSM is effective, it will free a lot of memory pages, > and total-free-committed becomes negative.
Thanks a lot for the information.
> > This was reported in https://bugzilla.redhat.com/show_bug.cgi?id=977758 > and the engine is using memFree reported by vdsm, which is more accurate. > > The API reports the old version due to backwards compatibility.
I just had a look at the bugzilla report and the RHEV documentation which says "Current memory in bytes used by the virtual machine.". So this means, the reported values are totally useless for monitoring memory usage of a virtual machine if KSM is active. I would expect to get the memory usage of a virtual machine and not how much memory is consumed on the hypervisor (this is pretty useless information for me).
Is it planned to report the memory usage in a virtual machine in the API as well?
Regards, René
> > Doron > > P.S. > René- thanks for helping!
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
OK, Thanks for the helpfull information. Learned something more again! Kind regards, Koen On Feb 12, 2014 5:22 PM, "Doron Fediuck" <dfediuck@redhat.com> wrote:
From: "René Koch" <rkoch@linuxland.at> To: "Koen Vanoppen" <vanoppen.koen@gmail.com> Cc: users@ovirt.org Sent: Wednesday, February 12, 2014 4:18:37 PM Subject: Re: [Users] Memory usage
On Wed, 2014-02-12 at 15:14 +0100, Koen Vanoppen wrote:
In The GUI, it says it's using 25% of the memory.
I guess it's the real value, right? The same happened for the memcached vm, someone reported to me - negative value in REST-API, but correct graph in oVirt webadmin GUI.
I fear I have no idea how this can happen - so maybe someone else can help you troubleshoot this issue.
2014-02-12 15:10 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>: Thanks for the quick respons, but there is no memcached running on that VM.
Kind regards
2014-02-12 15:06 GMT+01:00 René Koch <rkoch@linuxland.at>:
On Wed, 2014-02-12 at 14:55 +0100, Koen Vanoppen wrote: > Dear all, > > > When we monitor one of our machines, we noticed that there was one vm > that was constantly giving a error of memory usage. But when we took a > look at it, there is actually nothing wrong with it. Now we looked > furhter then that. We looked at the API of the machine and noticed > something very strange: > > <statistic >
----- Original Message ----- href="/api/vms/3b9aa245-75ff-42e8-b921-1c9ce61826bf/statistics/b7499508-c1c3-32f0-8174-c1783e57bb08"
id="b7499508-c1c3-32f0-8174-c1783e57bb08"><name>memory.used</name><description>Memory
used (agent)</description><values
type="INTEGER"><value><datum>-944892806</datum></value></values><type>GAUGE</type><unit>BYTES</unit>
> > > It's a negative...
Do you have memcached running in this vm?
I heard about this issue with memcached, but never tested memcached in my oVirt environment. You get the real usage value with memory.used = memory.installed + memory.used
Regards, René
> > > What could be the problem? > > > Kind regards, > > koen >
Guys, these values are usually a result of overcommitment mechanism usage. For example, if KSM is effective, it will free a lot of memory pages, and total-free-committed becomes negative.
This was reported in https://bugzilla.redhat.com/show_bug.cgi?id=977758 and the engine is using memFree reported by vdsm, which is more accurate.
The API reports the old version due to backwards compatibility.
Doron
P.S. René- thanks for helping!
participants (5)
-
david van zeebroeck -
Doron Fediuck -
Koen Vanoppen -
René Koch -
Sven Kieske