Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html -- Tomáš Golembiovský <tgolembi@redhat.com>
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi,
there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions).
There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory.
I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time.
Tomas
[1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE: [RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010
On 28 Aug 2017, at 13:37, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE:
we can either declare it deprecated or temporarily report 0 until the support propagates into qemu-ga I think best way forward is to just make it optionally available when ovirt-guest-agent reports it. I.e. users will get only part of the information when they run only qemu-ga, and that’s fine…
[RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010 _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 08/28/2017 01:42 PM, Michal Skrivanek wrote:
On 28 Aug 2017, at 13:37, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE:
we can either declare it deprecated or temporarily report 0 until the support propagates into qemu-ga
So there will be an interval of time where it will not be reported? Will that affect released versions of the system or it is just something that we will re-add before the next release? It is released versions of the system that I am concerned about. We should not do a release that breaks backwards compatibility. It is completely OK if we break it in between releases.
I think best way forward is to just make it optionally available when ovirt-guest-agent reports it. I.e. users will get only part of the information when they run only qemu-ga, and that’s fine…
[RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010
On 08/28/2017 01:57 PM, Juan Hernández wrote:
On 08/28/2017 01:42 PM, Michal Skrivanek wrote:
On 28 Aug 2017, at 13:37, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE:
we can either declare it deprecated or temporarily report 0 until the support propagates into qemu-ga
So there will be an interval of time where it will not be reported? Will that affect released versions of the system or it is just something that we will re-add before the next release? It is released versions of the system that I am concerned about. We should not do a release that breaks backwards compatibility. It is completely OK if we break it in between releases.
I think best way forward is to just make it optionally available when ovirt-guest-agent reports it. I.e. users will get only part of the information when they run only qemu-ga, and that’s fine…
Not sure I understand this. Currently, if I understand correctly, these values are only populated if the ovirt-guest-agent is installed? That is OK (may be worth documenting it). We plan to replace ovirt-guest-agent with qemu-ga in a mandatory way? Or will the user be able to use one or the other? If the user decides to use the ovirt-guest-agent, will we continue to report these statistics?
[RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010
On 28 Aug 2017, at 14:02, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:57 PM, Juan Hernández wrote:
On 08/28/2017 01:42 PM, Michal Skrivanek wrote:
On 28 Aug 2017, at 13:37, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE:
we can either declare it deprecated or temporarily report 0 until the support propagates into qemu-ga
So there will be an interval of time where it will not be reported? Will that affect released versions of the system or it is just something that we will re-add before the next release? It is released versions of the system that I am concerned about. We should not do a release that breaks backwards compatibility. It is completely OK if we break it in between releases.
I think best way forward is to just make it optionally available when ovirt-guest-agent reports it. I.e. users will get only part of the information when they run only qemu-ga, and that’s fine…
Not sure I understand this. Currently, if I understand correctly, these values are only populated if the ovirt-guest-agent is installed? That is OK (may be worth documenting it). We plan to replace ovirt-guest-agent with qemu-ga in a mandatory way? Or will the user be able to use one or the other?
both are mandatory today, though not really enforced, there’s just a “!” next to the VM in the VMs grid. The plan is to switch data retrieval to qemu-ga and make ovirt-ga optional (it’s going to be mandatory for specific purposes, like SSO)
If the user decides to use the ovirt-guest-agent, will we continue to report these statistics?
yes, I expect so Thanks, michal
[RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010
On 08/28/2017 02:05 PM, Michal Skrivanek wrote:
On 28 Aug 2017, at 14:02, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:57 PM, Juan Hernández wrote:
On 08/28/2017 01:42 PM, Michal Skrivanek wrote:
On 28 Aug 2017, at 13:37, Juan Hernández <jhernand@redhat.com> wrote:
On 08/28/2017 01:11 PM, Tomáš Golembiovský wrote:
Hi, there is ongoing effort to change how VDSM collects information about memory usage from guests. We used to use oVirt guest agent to get the statistics about free memory, swap usage, etc. This is going to change and we will use stats provided by VirtIO Balloon driver. [1] This should not break MOM nor Engine (current or previous versions). There is however small downside to this change. We will not be able to fetch information about buffers and caches on Linux guests anymore (mem_cached and mem_buffers in memoryStats). The corollary of this is that free memory reported by Engine will also no longer include those statistics and reported free memory will be really (only) free memory. I now there will be mixed feelings about this and we would like to get the information about buffers and caches back to the VDSM. Correctly via balloon driver statistics. [2] This effort will however take some time. Tomas [1] https://gerrit.ovirt.org/#/q/topic:memory-stats [2] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg05239.html
Note that this information is currently reported by the engine REST API, so it is part of its contract. Removing it breaks backwards compatibility and causes a regression in this RFE:
we can either declare it deprecated or temporarily report 0 until the support propagates into qemu-ga
So there will be an interval of time where it will not be reported? Will that affect released versions of the system or it is just something that we will re-add before the next release? It is released versions of the system that I am concerned about. We should not do a release that breaks backwards compatibility. It is completely OK if we break it in between releases.
I think best way forward is to just make it optionally available when ovirt-guest-agent reports it. I.e. users will get only part of the information when they run only qemu-ga, and that’s fine…
Not sure I understand this. Currently, if I understand correctly, these values are only populated if the ovirt-guest-agent is installed? That is OK (may be worth documenting it). We plan to replace ovirt-guest-agent with qemu-ga in a mandatory way? Or will the user be able to use one or the other?
both are mandatory today, though not really enforced, there’s just a “!” next to the VM in the VMs grid. The plan is to switch data retrieval to qemu-ga and make ovirt-ga optional (it’s going to be mandatory for specific purposes, like SSO)
If the user decides to use the ovirt-guest-agent, will we continue to report these statistics?
yes, I expect so
OK, in that case I think there is no problem in doing this change, from the API point of view. I'd just suggest to explicitly document in the specification of the API that these statistics will only be available if the ovirt-guest-agent is installed. I mean here: https://github.com/oVirt/ovirt-engine-api-model/blob/4b85ea1305d43ce9ec42219...
Thanks, michal
[RFE] Report guests Buffered/Cached memory https://bugzilla.redhat.com/1024010
participants (3)
-
Juan Hernández -
Michal Skrivanek -
Tomáš Golembiovský