[ovirt-devel] [libvirt] [RFC][scale] new API for querying domains stats

[Nir Soffer]

----- Original Message -----
> From: "Francesco Romani" <fromani at redhat.com>
> To: "Daniel P. Berrange" <berrange at redhat.com>
> Cc: libvir-list at redhat.com, devel at ovirt.org
> Sent: Wednesday, July 2, 2014 6:41:28 PM
> Subject: Re: [ovirt-devel] [libvirt] [RFC][scale] new API for querying	domains stats
> 
> ----- Original Message -----
> > From: "Daniel P. Berrange" <berrange at redhat.com>
> > To: "Francesco Romani" <fromani at redhat.com>
> > Cc: libvir-list at redhat.com
> > Sent: Tuesday, July 1, 2014 10:35:21 AM
> > Subject: Re: [libvirt] [RFC][scale] new API for querying domains stats
> > 
> 
> [...]
> > > We [in VDSM] currently use these APIs for our sempling:
> > >   virDomainBlockInfo
> > >   virDomainGetInfo
> > >   virDomainGetCPUStats
> > >   virDomainBlockStats
> > >   virDomainBlockStatsFlags
> > >   virDomainInterfaceStats
> > >   virDomainGetVcpusFlags
> > >   virDomainGetMetadata
> > 
> > Why do you need to call virDomainGetMetadata so often ? That merely
> > contains
> > a opaque data blob that can only have come from VDSM itself, so I'm
> > surprised
> > you need to call that at all frequently.
> 
> We store some QoS info in the domain metadata. Actually we can elide this API
> call
> from the list and fix our coude to make smarter use of it.
> 
> > >   please note that we are much more concerned about thread reduction then
> > >   about performance numbers. We had report of thread number becoming a
> > >   real harm, while performance so far is not yet a concern
> > >   (https://bugzilla.redhat.com/show_bug.cgi?id=1102147#c54)
> > > 
> > > * bulk APIs for querying domain stats
> > > (https://bugzilla.redhat.com/show_bug.cgi?id=1113116)
> > >   would be really welcome as well. It is quite independent from the
> > >   previous bullet point
> > >   and would help us greatly with scale.
> > 
> > If we did the first bullet point, we'd be adding another ~10 APIs for
> > async variants. If we then did the second bullet point we'd be adding
> > another ~10 APIs for bulk querying. So while you're right that they
> > are independent, it would be desirable to address them both at the
> > same time, so we only need to add 10 new APIs in total, not 20.
> 
> I'm fine with this approach.
> 
> 
> > For the async API design, I could see two potential designs
> > 
> > 1. A custom callback to run per API
> > 
> >      typedef (void)(*virDomainBlockInfoCallback)(virDomainPtr dom,
> >                                                  bool isError,
> >                                                  virDomainBlockInfoPtr
> >                                                  info,
> >                                                  void *opaque);
> > 
> >     int virDomainGetBlockInfoAsync(virDomainPtr dom,
> >                                    const char *disk,
> >                                    virDomainBlockInfoCallback cb,
> >                                    void *opaque,
> >                                    unsigned int flags);
> > 
> > 
> > 2. A standard callback and a pair of APIs
> > 
> >      typedef void *virDomainAsyncResult;
> >      typedef (void)(*virDomainAsyncCallback)(virDomainPtr dom,
> >                                              virDomainAsyncResult res);
> > 
> >    void virDomainGetBlockInfoAsync(virDomainPtr dom,
> >                                    const char *disk,
> >                                    virDomainBlockInfoCallback cb,
> >                                    void *opaque,
> >                                    unsigned int flags);
> >    int virDomainGetBlockInfoFinish(virDomainPtr dom,
> >                                   virDomainAsyncResult res,
> >                                   virDomainBlockInfoPtr info);
> > 
> > This second approach is the way GIO works (see example in this page
> > https://developer.gnome.org/gio/stable/GAsyncResult.html ). The main
> > difference between them really is probably the way you get error
> > reporting from the APIs. In the first example, libvirt would raise
> > an error before it invoked the callback, with isError set to True.
> > In the second example, the Finish() func would raise the error and
> > return -1.
> 
> I need to check in deeper detail and sync up with other VDSM developers,
> but I have a feel that the first approach is a bit easier for VDSM to
> consume.

The first approach looks simpler - I assume that on error will get the
callback with isError set to True, and we can get the error details 
within the callback?

I would like to see an example of client code using this api in both
success and error cases.

Nir