From fromani at redhat.com Wed Feb 22 15:57:10 2017 Content-Type: multipart/mixed; boundary="===============5928323909172622551==" MIME-Version: 1.0 From: Francesco Romani To: devel at ovirt.org Subject: Re: [ovirt-devel] [monitoring][collectd] the collectd virt plugin is now on par with Vdsm needs Date: Wed, 22 Feb 2017 16:57:07 +0100 Message-ID: In-Reply-To: CACKMAy9CwXBZpJSOXqpfKg8j4O7hGn4qR94VXAKyWxtGroUD8w@mail.gmail.com --===============5928323909172622551== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------D9BDB513741273D7BA913655 Content-Type: text/plain; charset=3Dutf-8 Content-Transfer-Encoding: 7bit On 02/21/2017 11:55 PM, Yaniv Dary wrote: > > > Yaniv Dary > Technical Product Manager > Red Hat Israel Ltd. > 34 Jerusalem Road > Building A, 4th floor > Ra'anana, Israel 4350109 > > Tel : +972 (9) 7692306 > 8272306 > Email: ydary(a)redhat.com > IRC : ydary > > On Feb 21, 2017 13:06, "Francesco Romani" > wrote: > > Hello everyone, > > > in the last weeks I've been submitting PRs to collectd upstream, to > bring the virt plugin up to date with Vdsm and oVirt needs. > > Previously, the collectd virt plugin reported only a subset of metrics > oVirt uses. > > In current collectd master, the collectd virt plugin provides all the > data Vdsm (thus Engine) needs. This means that it is now > > possible for Vdsm or Engine to query collectd, not Vdsm/libvirt, and > have the same data. > > > There are only two caveats: > > 1. it is yet to be seen which version of collectd will ship all those > enhancements > > 2. collectd *intentionally* report metrics as rates, not as absolute > values as Vdsm does. This may be one issue in presence of > restarts/data > loss in the link between collectd and the metrics store. > > > How does this work? = > If we want to show memory usage over time for example, we need to have > the usage, not the rate. = > How would this be reported? I was imprecise, my fault. Let me retry: collectd intentionally report quite a lot of metrics we care about as rates, not as absolute values. Memory is actually ok fine. a0/virt/disk_octets-hdc -> rate a0/virt/disk_octets-vda a0/virt/disk_ops-hdc -> rate a0/virt/disk_ops-vda a0/virt/disk_time-hdc -> rate a0/virt/disk_time-vda a0/virt/if_dropped-vnet0 -> rate a0/virt/if_errors-vnet0 -> rate a0/virt/if_octets-vnet0 -> rate a0/virt/if_packets-vnet0 -> rate a0/virt/memory-actual_balloon -> absolute a0/virt/memory-rss -> absolute a0/virt/memory-total -> absolute a0/virt/ps_cputime -> rate a0/virt/total_requests-flush-hdc -> rate a0/virt/total_requests-flush-vda a0/virt/total_time_in_ms-flush-hdc -> rate a0/virt/total_time_in_ms-flush-vda a0/virt/virt_cpu_total -> rate a0/virt/virt_vcpu-0 -> rate a0/virt/virt_vcpu-1 collectd "just" reports the changes since the last sampling. I'm not sure which is the best way to handle that; I've sent a mail to collectd list some time ago, no answer so far. -- = Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani --------------D9BDB513741273D7BA913655 Content-Type: text/html; charset=3Dutf-8 Content-Transfer-Encoding: 8bit On 02/21/2017 11:55 PM, Yaniv Dary wrote:


Yaniv Dary
Technical Product Manager
Red Hat Israel Ltd.
34 Jerusalem Road
Building A, 4th floor
Ra'anana, Israel 4350109

Tel : +972 (9) 7692306
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 8272306
Email: ydary(a)redhat.com
IRC : ydary

On Feb 21, 2017 13:06, "Francesco Romani" <fromani(a)redhat.com> wrote:
Hello everyone,


in the last weeks I've been submitting PRs to collectd upstream, to
bring the virt plugin up to date with Vdsm and oVirt needs.

Previously, the collectd virt plugin reported only a subset of metrics
oVirt uses.

In current collectd master, the collectd virt plugin provides all the
data Vdsm (thus Engine) needs. This means that it is now

possible for Vdsm or Engine to query collectd, not Vdsm/libvirt, and
have the same data.


There are only two caveats:

1. it is yet to be seen which version of collectd will ship all those
enhancements

2. collectd *intentionally* report metrics as rates, not as absolute
values as Vdsm does. This may be one issue in presence of restarts/data
loss in the link between collectd and the metrics store.

How does this work?=C2=A0
If we want to show memory usage over time for example, we need to have the usage, not the rate.=C2=A0
How would this be reported?

I was imprecise, my fault.

Let me retry:
collectd intentionally report quite a lot of metrics we care about as rates, not as absolute values.
Memory is actually ok fine.

=C2=A0 a0/virt/disk_octets-hdc -> rate
=C2=A0 a0/virt/disk_octets-vda
=C2=A0 a0/virt/disk_ops-hdc -> rate
=C2=A0 a0/virt/disk_ops-vda
=C2=A0 a0/virt/disk_time-hdc -> rate
=C2=A0 a0/virt/disk_time-vda
=C2=A0 a0/virt/if_dropped-vnet0 -> rate
=C2=A0 a0/virt/if_errors-vnet0 -> rate
=C2=A0 a0/virt/if_octets-vnet0 -> rate
=C2=A0 a0/virt/if_packets-vnet0 -> rate
=C2=A0 a0/virt/memory-actual_balloon -> absolute
=C2=A0 a0/virt/memory-rss -> absolute
=C2=A0 a0/virt/memory-total -> absolute
=C2=A0 a0/virt/ps_cputime -> rate
=C2=A0 a0/virt/total_requests-flush-hdc ->=C2=A0 rate
=C2=A0 a0/virt/total_requests-flush-vda
=C2=A0 a0/virt/total_time_in_ms-flush-hdc -> rate
=C2=A0 a0/virt/total_time_in_ms-flush-vda
=C2=A0 a0/virt/virt_cpu_total -> rate
=C2=A0 a0/virt/virt_vcpu-0 -> rate
=C2=A0 a0/virt/virt_vcpu-1

collectd "just" reports the changes since the last sampling. I'm not sure which is the best way to handle that; I've sent a mail to collectd list some time ago, no answer so far.



-- =

Francesco Romani
Red Hat Engineering Virtualization R & D
IRC: fromani
--------------D9BDB513741273D7BA913655-- --===============5928323909172622551== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS1EOUJEQjUxMzc0MTI3M0Q3QkE5MTM2NTUKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXV0Zi04CkNvbnRlbnQtVHJhbnNmZXItRW5jb2Rpbmc6IDdiaXQKCk9uIDAyLzIxLzIwMTcg MTE6NTUgUE0sIFlhbml2IERhcnkgd3JvdGU6Cj4KPgo+IFlhbml2IERhcnkKPiBUZWNobmljYWwg UHJvZHVjdCBNYW5hZ2VyCj4gUmVkIEhhdCBJc3JhZWwgTHRkLgo+IDM0IEplcnVzYWxlbSBSb2Fk Cj4gQnVpbGRpbmcgQSwgNHRoIGZsb29yCj4gUmEnYW5hbmEsIElzcmFlbCA0MzUwMTA5Cj4KPiBU ZWwgOiArOTcyICg5KSA3NjkyMzA2Cj4gICAgICAgICA4MjcyMzA2Cj4gRW1haWw6IHlkYXJ5QHJl ZGhhdC5jb20gPG1haWx0bzp5ZGFyeUByZWRoYXQuY29tPgo+IElSQyA6IHlkYXJ5Cj4KPiBPbiBG ZWIgMjEsIDIwMTcgMTM6MDYsICJGcmFuY2VzY28gUm9tYW5pIiA8ZnJvbWFuaUByZWRoYXQuY29t Cj4gPG1haWx0bzpmcm9tYW5pQHJlZGhhdC5jb20+PiB3cm90ZToKPgo+ICAgICBIZWxsbyBldmVy eW9uZSwKPgo+Cj4gICAgIGluIHRoZSBsYXN0IHdlZWtzIEkndmUgYmVlbiBzdWJtaXR0aW5nIFBS cyB0byBjb2xsZWN0ZCB1cHN0cmVhbSwgdG8KPiAgICAgYnJpbmcgdGhlIHZpcnQgcGx1Z2luIHVw IHRvIGRhdGUgd2l0aCBWZHNtIGFuZCBvVmlydCBuZWVkcy4KPgo+ICAgICBQcmV2aW91c2x5LCB0 aGUgY29sbGVjdGQgdmlydCBwbHVnaW4gcmVwb3J0ZWQgb25seSBhIHN1YnNldCBvZiBtZXRyaWNz Cj4gICAgIG9WaXJ0IHVzZXMuCj4KPiAgICAgSW4gY3VycmVudCBjb2xsZWN0ZCBtYXN0ZXIsIHRo ZSBjb2xsZWN0ZCB2aXJ0IHBsdWdpbiBwcm92aWRlcyBhbGwgdGhlCj4gICAgIGRhdGEgVmRzbSAo dGh1cyBFbmdpbmUpIG5lZWRzLiBUaGlzIG1lYW5zIHRoYXQgaXQgaXMgbm93Cj4KPiAgICAgcG9z c2libGUgZm9yIFZkc20gb3IgRW5naW5lIHRvIHF1ZXJ5IGNvbGxlY3RkLCBub3QgVmRzbS9saWJ2 aXJ0LCBhbmQKPiAgICAgaGF2ZSB0aGUgc2FtZSBkYXRhLgo+Cj4KPiAgICAgVGhlcmUgYXJlIG9u bHkgdHdvIGNhdmVhdHM6Cj4KPiAgICAgMS4gaXQgaXMgeWV0IHRvIGJlIHNlZW4gd2hpY2ggdmVy c2lvbiBvZiBjb2xsZWN0ZCB3aWxsIHNoaXAgYWxsIHRob3NlCj4gICAgIGVuaGFuY2VtZW50cwo+ Cj4gICAgIDIuIGNvbGxlY3RkICppbnRlbnRpb25hbGx5KiByZXBvcnQgbWV0cmljcyBhcyByYXRl cywgbm90IGFzIGFic29sdXRlCj4gICAgIHZhbHVlcyBhcyBWZHNtIGRvZXMuIFRoaXMgbWF5IGJl IG9uZSBpc3N1ZSBpbiBwcmVzZW5jZSBvZgo+ICAgICByZXN0YXJ0cy9kYXRhCj4gICAgIGxvc3Mg aW4gdGhlIGxpbmsgYmV0d2VlbiBjb2xsZWN0ZCBhbmQgdGhlIG1ldHJpY3Mgc3RvcmUuCj4KPgo+ IEhvdyBkb2VzIHRoaXMgd29yaz8gCj4gSWYgd2Ugd2FudCB0byBzaG93IG1lbW9yeSB1c2FnZSBv dmVyIHRpbWUgZm9yIGV4YW1wbGUsIHdlIG5lZWQgdG8gaGF2ZQo+IHRoZSB1c2FnZSwgbm90IHRo ZSByYXRlLiAKPiBIb3cgd291bGQgdGhpcyBiZSByZXBvcnRlZD8KCkkgd2FzIGltcHJlY2lzZSwg bXkgZmF1bHQuCgpMZXQgbWUgcmV0cnk6CmNvbGxlY3RkIGludGVudGlvbmFsbHkgcmVwb3J0IHF1 aXRlIGEgbG90IG9mIG1ldHJpY3Mgd2UgY2FyZSBhYm91dCBhcwpyYXRlcywgbm90IGFzIGFic29s dXRlIHZhbHVlcy4KTWVtb3J5IGlzIGFjdHVhbGx5IG9rIGZpbmUuCgogIGEwL3ZpcnQvZGlza19v Y3RldHMtaGRjIC0+IHJhdGUKICBhMC92aXJ0L2Rpc2tfb2N0ZXRzLXZkYQogIGEwL3ZpcnQvZGlz a19vcHMtaGRjIC0+IHJhdGUKICBhMC92aXJ0L2Rpc2tfb3BzLXZkYQogIGEwL3ZpcnQvZGlza190 aW1lLWhkYyAtPiByYXRlCiAgYTAvdmlydC9kaXNrX3RpbWUtdmRhCiAgYTAvdmlydC9pZl9kcm9w cGVkLXZuZXQwIC0+IHJhdGUKICBhMC92aXJ0L2lmX2Vycm9ycy12bmV0MCAtPiByYXRlCiAgYTAv dmlydC9pZl9vY3RldHMtdm5ldDAgLT4gcmF0ZQogIGEwL3ZpcnQvaWZfcGFja2V0cy12bmV0MCAt PiByYXRlCiAgYTAvdmlydC9tZW1vcnktYWN0dWFsX2JhbGxvb24gLT4gYWJzb2x1dGUKICBhMC92 aXJ0L21lbW9yeS1yc3MgLT4gYWJzb2x1dGUKICBhMC92aXJ0L21lbW9yeS10b3RhbCAtPiBhYnNv bHV0ZQogIGEwL3ZpcnQvcHNfY3B1dGltZSAtPiByYXRlCiAgYTAvdmlydC90b3RhbF9yZXF1ZXN0 cy1mbHVzaC1oZGMgLT4gIHJhdGUKICBhMC92aXJ0L3RvdGFsX3JlcXVlc3RzLWZsdXNoLXZkYQog IGEwL3ZpcnQvdG90YWxfdGltZV9pbl9tcy1mbHVzaC1oZGMgLT4gcmF0ZQogIGEwL3ZpcnQvdG90 YWxfdGltZV9pbl9tcy1mbHVzaC12ZGEKICBhMC92aXJ0L3ZpcnRfY3B1X3RvdGFsIC0+IHJhdGUK ICBhMC92aXJ0L3ZpcnRfdmNwdS0wIC0+IHJhdGUKICBhMC92aXJ0L3ZpcnRfdmNwdS0xCgpjb2xs ZWN0ZCAianVzdCIgcmVwb3J0cyB0aGUgY2hhbmdlcyBzaW5jZSB0aGUgbGFzdCBzYW1wbGluZy4g SSdtIG5vdApzdXJlIHdoaWNoIGlzIHRoZSBiZXN0IHdheSB0byBoYW5kbGUgdGhhdDsgSSd2ZSBz ZW50IGEgbWFpbCB0byBjb2xsZWN0ZApsaXN0IHNvbWUgdGltZSBhZ28sIG5vIGFuc3dlciBzbyBm YXIuCgoKCi0tIApGcmFuY2VzY28gUm9tYW5pClJlZCBIYXQgRW5naW5lZXJpbmcgVmlydHVhbGl6 YXRpb24gUiAmIEQKSVJDOiBmcm9tYW5pCgoKLS0tLS0tLS0tLS0tLS1EOUJEQjUxMzc0MTI3M0Q3 QkE5MTM2NTUKQ29udGVudC1UeXBlOiB0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgKQ29udGVudC1U cmFuc2Zlci1FbmNvZGluZzogOGJpdAoKPGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBjb250ZW50 PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSI+CiAg PC9oZWFkPgogIDxib2R5IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPgogICAgT24g MDIvMjEvMjAxNyAxMTo1NSBQTSwgWWFuaXYgRGFyeSB3cm90ZTo8YnI+CiAgICA8YmxvY2txdW90 ZQpjaXRlPSJtaWQ6Q0FDS01BeTlDd1hCWnBKU09YcXBmS2c4ajRPN2hHbjRxUjk0VlhBS3lXeHRH cm9VRDh3QG1haWwuZ21haWwuY29tIgogICAgICB0eXBlPSJjaXRlIj4KICAgICAgPGRpdiBkaXI9 ImF1dG8iPgogICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgPGJyPgogICAgICAgICAgPGRpdiBk YXRhLXNtYXJ0bWFpbD0iZ21haWxfc2lnbmF0dXJlIj5ZYW5pdiBEYXJ5PGJyPgogICAgICAgICAg ICBUZWNobmljYWwgUHJvZHVjdCBNYW5hZ2VyPGJyPgogICAgICAgICAgICBSZWQgSGF0IElzcmFl bCBMdGQuPGJyPgogICAgICAgICAgICAzNCBKZXJ1c2FsZW0gUm9hZDxicj4KICAgICAgICAgICAg QnVpbGRpbmcgQSwgNHRoIGZsb29yPGJyPgogICAgICAgICAgICBSYSdhbmFuYSwgSXNyYWVsIDQz NTAxMDk8YnI+CiAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgVGVsIDogKzk3MiAoOSkgNzY5 MjMwNjxicj4KICAgICAgICAgICAgwqDCoMKgwqDCoMKgwqAgODI3MjMwNjxicj4KICAgICAgICAg ICAgRW1haWw6IDxhIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSIKICAgICAgICAgICAgICBocmVmPSJt YWlsdG86eWRhcnlAcmVkaGF0LmNvbSI+eWRhcnlAcmVkaGF0LmNvbTwvYT48YnI+CiAgICAgICAg ICAgIElSQyA6IHlkYXJ5PC9kaXY+CiAgICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+ PGJyPgogICAgICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gRmViIDIxLCAyMDE3 IDEzOjA2LCAiRnJhbmNlc2NvCiAgICAgICAgICAgICAgUm9tYW5pIiAmbHQ7PGEgbW96LWRvLW5v dC1zZW5kPSJ0cnVlIgogICAgICAgICAgICAgICAgaHJlZj0ibWFpbHRvOmZyb21hbmlAcmVkaGF0 LmNvbSI+ZnJvbWFuaUByZWRoYXQuY29tPC9hPiZndDsKICAgICAgICAgICAgICB3cm90ZTo8YnIg dHlwZT0iYXR0cmlidXRpb24iPgogICAgICAgICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJxdW90 ZSIgc3R5bGU9Im1hcmdpbjowIDAgMAogICAgICAgICAgICAgICAgLjhleDtib3JkZXItbGVmdDox cHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij5IZWxsbwogICAgICAgICAgICAgICAgZXZl cnlvbmUsPGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAg ICAgICAgICAgICAgaW4gdGhlIGxhc3Qgd2Vla3MgSSd2ZSBiZWVuIHN1Ym1pdHRpbmcgUFJzIHRv IGNvbGxlY3RkCiAgICAgICAgICAgICAgICB1cHN0cmVhbSwgdG88YnI+CiAgICAgICAgICAgICAg ICBicmluZyB0aGUgdmlydCBwbHVnaW4gdXAgdG8gZGF0ZSB3aXRoIFZkc20gYW5kIG9WaXJ0CiAg ICAgICAgICAgICAgICBuZWVkcy48YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgICBQcmV2aW91c2x5LCB0aGUgY29sbGVjdGQgdmlydCBwbHVnaW4gcmVwb3J0ZWQgb25seSBh CiAgICAgICAgICAgICAgICBzdWJzZXQgb2YgbWV0cmljczxicj4KICAgICAgICAgICAgICAgIG9W aXJ0IHVzZXMuPGJyPgogICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgSW4gY3Vy cmVudCBjb2xsZWN0ZCBtYXN0ZXIsIHRoZSBjb2xsZWN0ZCB2aXJ0IHBsdWdpbgogICAgICAgICAg ICAgICAgcHJvdmlkZXMgYWxsIHRoZTxicj4KICAgICAgICAgICAgICAgIGRhdGEgVmRzbSAodGh1 cyBFbmdpbmUpIG5lZWRzLiBUaGlzIG1lYW5zIHRoYXQgaXQgaXMgbm93PGJyPgogICAgICAgICAg ICAgICAgPGJyPgogICAgICAgICAgICAgICAgcG9zc2libGUgZm9yIFZkc20gb3IgRW5naW5lIHRv IHF1ZXJ5IGNvbGxlY3RkLCBub3QKICAgICAgICAgICAgICAgIFZkc20vbGlidmlydCwgYW5kPGJy PgogICAgICAgICAgICAgICAgaGF2ZSB0aGUgc2FtZSBkYXRhLjxicj4KICAgICAgICAgICAgICAg IDxicj4KICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIFRoZXJlIGFyZSBvbmx5 IHR3byBjYXZlYXRzOjxicj4KICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIDEu IGl0IGlzIHlldCB0byBiZSBzZWVuIHdoaWNoIHZlcnNpb24gb2YgY29sbGVjdGQgd2lsbAogICAg ICAgICAgICAgICAgc2hpcCBhbGwgdGhvc2U8YnI+CiAgICAgICAgICAgICAgICBlbmhhbmNlbWVu dHM8YnI+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAyLiBjb2xsZWN0ZCAq aW50ZW50aW9uYWxseSogcmVwb3J0IG1ldHJpY3MgYXMgcmF0ZXMsIG5vdAogICAgICAgICAgICAg ICAgYXMgYWJzb2x1dGU8YnI+CiAgICAgICAgICAgICAgICB2YWx1ZXMgYXMgVmRzbSBkb2VzLiBU aGlzIG1heSBiZSBvbmUgaXNzdWUgaW4gcHJlc2VuY2UKICAgICAgICAgICAgICAgIG9mIHJlc3Rh cnRzL2RhdGE8YnI+CiAgICAgICAgICAgICAgICBsb3NzIGluIHRoZSBsaW5rIGJldHdlZW4gY29s bGVjdGQgYW5kIHRoZSBtZXRyaWNzIHN0b3JlLjxicj4KICAgICAgICAgICAgICA8L2Jsb2NrcXVv dGU+CiAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgPC9kaXY+CiAgICAgICAgPC9kaXY+CiAg ICAgICAgPGRpdiBkaXI9ImF1dG8iPjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2IGRp cj0iYXV0byI+SG93IGRvZXMgdGhpcyB3b3JrP8KgPC9kaXY+CiAgICAgICAgPGRpdiBkaXI9ImF1 dG8iPklmIHdlIHdhbnQgdG8gc2hvdyBtZW1vcnkgdXNhZ2Ugb3ZlciB0aW1lIGZvcgogICAgICAg ICAgZXhhbXBsZSwgd2UgbmVlZCB0byBoYXZlIHRoZSB1c2FnZSwgbm90IHRoZSByYXRlLsKgPC9k aXY+CiAgICAgICAgPGRpdiBkaXI9ImF1dG8iPkhvdyB3b3VsZCB0aGlzIGJlIHJlcG9ydGVkPyA8 YnI+CiAgICAgICAgPC9kaXY+CiAgICAgIDwvZGl2PgogICAgPC9ibG9ja3F1b3RlPgogICAgPGJy PgogICAgSSB3YXMgaW1wcmVjaXNlLCBteSBmYXVsdC48YnI+CiAgICA8YnI+CiAgICBMZXQgbWUg cmV0cnk6PGJyPgogICAgY29sbGVjdGQgaW50ZW50aW9uYWxseSByZXBvcnQgcXVpdGUgYSBsb3Qg b2YgbWV0cmljcyB3ZSBjYXJlIGFib3V0CiAgICBhcyByYXRlcywgbm90IGFzIGFic29sdXRlIHZh bHVlcy48YnI+CiAgICBNZW1vcnkgaXMgYWN0dWFsbHkgb2sgZmluZS48YnI+CiAgICA8YnI+CiAg ICDCoCBhMC92aXJ0L2Rpc2tfb2N0ZXRzLWhkYyAtJmd0OyByYXRlPGJyPgogICAgwqAgYTAvdmly dC9kaXNrX29jdGV0cy12ZGE8YnI+CiAgICDCoCBhMC92aXJ0L2Rpc2tfb3BzLWhkYyAtJmd0OyBy YXRlPGJyPgogICAgwqAgYTAvdmlydC9kaXNrX29wcy12ZGE8YnI+CiAgICDCoCBhMC92aXJ0L2Rp c2tfdGltZS1oZGMgLSZndDsgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQvZGlza190aW1lLXZkYTxi cj4KICAgIMKgIGEwL3ZpcnQvaWZfZHJvcHBlZC12bmV0MCAtJmd0OyByYXRlPGJyPgogICAgwqAg YTAvdmlydC9pZl9lcnJvcnMtdm5ldDAgLSZndDsgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQvaWZf b2N0ZXRzLXZuZXQwIC0mZ3Q7IHJhdGU8YnI+CiAgICDCoCBhMC92aXJ0L2lmX3BhY2tldHMtdm5l dDAgLSZndDsgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQvbWVtb3J5LWFjdHVhbF9iYWxsb29uIC0m Z3Q7IGFic29sdXRlPGJyPgogICAgwqAgYTAvdmlydC9tZW1vcnktcnNzIC0mZ3Q7IGFic29sdXRl PGJyPgogICAgwqAgYTAvdmlydC9tZW1vcnktdG90YWwgLSZndDsgYWJzb2x1dGU8YnI+CiAgICDC oCBhMC92aXJ0L3BzX2NwdXRpbWUgLSZndDsgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQvdG90YWxf cmVxdWVzdHMtZmx1c2gtaGRjIC0mZ3Q7wqAgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQvdG90YWxf cmVxdWVzdHMtZmx1c2gtdmRhPGJyPgogICAgwqAgYTAvdmlydC90b3RhbF90aW1lX2luX21zLWZs dXNoLWhkYyAtJmd0OyByYXRlPGJyPgogICAgwqAgYTAvdmlydC90b3RhbF90aW1lX2luX21zLWZs dXNoLXZkYTxicj4KICAgIMKgIGEwL3ZpcnQvdmlydF9jcHVfdG90YWwgLSZndDsgcmF0ZTxicj4K ICAgIMKgIGEwL3ZpcnQvdmlydF92Y3B1LTAgLSZndDsgcmF0ZTxicj4KICAgIMKgIGEwL3ZpcnQv dmlydF92Y3B1LTE8YnI+CiAgICA8YnI+CiAgICBjb2xsZWN0ZCAianVzdCIgcmVwb3J0cyB0aGUg Y2hhbmdlcyBzaW5jZSB0aGUgbGFzdCBzYW1wbGluZy4gSSdtIG5vdAogICAgc3VyZSB3aGljaCBp cyB0aGUgYmVzdCB3YXkgdG8gaGFuZGxlIHRoYXQ7IEkndmUgc2VudCBhIG1haWwgdG8KICAgIGNv bGxlY3RkIGxpc3Qgc29tZSB0aW1lIGFnbywgbm8gYW5zd2VyIHNvIGZhci48YnI+CiAgICA8YnI+ CiAgICA8YnI+CiAgICA8YnI+CiAgICA8cHJlIGNsYXNzPSJtb3otc2lnbmF0dXJlIiBjb2xzPSI3 MiI+LS0gCkZyYW5jZXNjbyBSb21hbmkKUmVkIEhhdCBFbmdpbmVlcmluZyBWaXJ0dWFsaXphdGlv biBSICZhbXA7IEQKSVJDOiBmcm9tYW5pPC9wcmU+CiAgPC9ib2R5Pgo8L2h0bWw+CgotLS0tLS0t LS0tLS0tLUQ5QkRCNTEzNzQxMjczRDdCQTkxMzY1NS0tCg== --===============5928323909172622551==--