[vdsm] new internal stable modules + proposal
Hi, in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/ - stable API - (thus) safe to use across verticals the planned moves are: lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py lib/vdsm/virt/api.py -> lib/vdsm/common/api.py Question is if those modules should go under common/ or under another subdirectory, maybe infra? Lastly, i have a proposal about better handling of those modules. First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations? for example something like __API__ = {} near the top of the module if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement). Like: __API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" } We could refine further this concept if we like it. The idea is to be lightweight as possible while carrying all the information we need. Comments welcome as usual bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
On Tue, Mar 29, 2016 at 6:12 PM, Francesco Romani <fromani@redhat.com> wrote:
Hi,
in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/
- stable API - (thus) safe to use across verticals
the planned moves are:
lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py
+1 I think we want to move also the executor, used by virt and jsonrpc. We plan to move all storage jobs to the executor.
lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py
Is it used outside of virt?
lib/vdsm/virt/api.py -> lib/vdsm/common/api.py
+1
Question is if those modules should go under common/ or under another subdirectory, maybe infra?
I would not add infra package, common *is* infra.
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
Since this is internal api, I don't think we need this info. The guarantee you have when you use such module is that it will never break your code. If someone one is changing such module, it must change and test the code using it. Maybe the "standard" __author__ ? Not sure it will help much, using git shortlog path/to/module is probably more useful when you look for someone to review a patch for some module.
We could refine further this concept if we like it. The idea is to be lightweight as possible while carrying all the information we need.
Adding nothing is the most lightweight way :-) Nir
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "devel" <devel@ovirt.org> Sent: Tuesday, March 29, 2016 6:00:56 PM Subject: Re: [ovirt-devel] [vdsm] new internal stable modules + proposal
On Tue, Mar 29, 2016 at 6:12 PM, Francesco Romani <fromani@redhat.com> wrote:
Hi,
in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/
- stable API - (thus) safe to use across verticals
the planned moves are:
lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py
+1
I think we want to move also the executor, used by virt and jsonrpc. We plan to move all storage jobs to the executor.
lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py
Is it used outside of virt?
Yes, sorry, I meant executor.py. Periodic.py will stay in virt for a little more.
lib/vdsm/virt/api.py -> lib/vdsm/common/api.py
+1
Question is if those modules should go under common/ or under another subdirectory, maybe infra?
I would not add infra package, common *is* infra.
OK, to explain further we have already lib/vdsm/common/network BTW I'm fine using lib/vdsm/common/ directly. The move will be done this friday unless anyone objects. -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "devel" <devel@ovirt.org> Sent: Tuesday, March 29, 2016 6:00:56 PM Subject: Re: [ovirt-devel] [vdsm] new internal stable modules + proposal [...]
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
Since this is internal api, I don't think we need this info. The guarantee you have when you use such module is that it will never break your code. If someone one is changing such module, it must change and test the code using it.
Maybe the "standard" __author__ ?
Yes, maybe just __author__ , __deprecated__ and perhaps __status__. I think __deprecated__ would be fit for the task. (take from e.g. http://stackoverflow.com/questions/1523427/what-is-the-common-header-format-... ) -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani
On Wed, Mar 30, 2016 at 9:20 AM, Francesco Romani <fromani@redhat.com> wrote:
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Francesco Romani" <fromani@redhat.com> Cc: "devel" <devel@ovirt.org> Sent: Tuesday, March 29, 2016 6:00:56 PM Subject: Re: [ovirt-devel] [vdsm] new internal stable modules + proposal [...]
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
Since this is internal api, I don't think we need this info. The guarantee you have when you use such module is that it will never break your code. If someone one is changing such module, it must change and test the code using it.
Maybe the "standard" __author__ ?
Yes, maybe just __author__ , __deprecated__ and perhaps __status__.
I think __deprecated__ would be fit for the task.
Do we have deprecated internal modules?
(take from e.g. http://stackoverflow.com/questions/1523427/what-is-the-common-header-format-... )
-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 29/03/16 11:12 -0400, Francesco Romani wrote:
Hi,
in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/
- stable API - (thus) safe to use across verticals
the planned moves are:
lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py lib/vdsm/virt/api.py -> lib/vdsm/common/api.py
Question is if those modules should go under common/ or under another subdirectory, maybe infra?
Hi. I agree that some modules should be kept (and advertised) as stable. The name 'common' is more suited for such package as 'infra' is very ovirt specific and could hurt readability and navigation of the project. It does make sense to structure the project as follows: lib/vdsm/common lib/vdsm/virt/common lib/vdsm/storage/common lib/vdsm/network/common Taking extra care taken to avoid 'common junkyard' situation and making sure that the scope of common corresponds to it's package. List of modules that I consider public (mostly author/main contributor of these lately): lib/vdsm/cpuarch.py lib/vdsm/cpuinfo.py lib/vdsm/hostdev.py lib/vdsm/machinetype.py lib/vdsm/numa.py lib/vdsm/osinfo.py lib/vdsm/udevadm.py
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
We could refine further this concept if we like it. The idea is to be lightweight as possible while carrying all the information we need.
I agree about keeping global metadata in that way, except ideally using docstrings for better readability (e.g. interactive help()). To better reflect the needs for granular deprecation and based on our private discussion on this subject, I'd like to see at least @deprecated(target_removal_version) decorator. On top of that, we need properly documented public API and deprecation process.
Comments welcome as usual
bests,
-- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Tue, Mar 29, 2016 at 7:44 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 29/03/16 11:12 -0400, Francesco Romani wrote:
Hi,
in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/
- stable API - (thus) safe to use across verticals
the planned moves are:
lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py lib/vdsm/virt/api.py -> lib/vdsm/common/api.py
Question is if those modules should go under common/ or under another subdirectory, maybe infra?
Hi.
I agree that some modules should be kept (and advertised) as stable.
The name 'common' is more suited for such package as 'infra' is very ovirt specific and could hurt readability and navigation of the project. It does make sense to structure the project as follows:
lib/vdsm/common lib/vdsm/virt/common lib/vdsm/storage/common lib/vdsm/network/common
This creates a dependency between the packages. The goal is to have packages that do not depend on each other except the common package, which is library code used by other parts.
Taking extra care taken to avoid 'common junkyard' situation and making sure that the scope of common corresponds to it's package.
List of modules that I consider public (mostly author/main contributor of these lately):
lib/vdsm/cpuarch.py lib/vdsm/cpuinfo.py lib/vdsm/hostdev.py lib/vdsm/machinetype.py lib/vdsm/numa.py lib/vdsm/osinfo.py lib/vdsm/udevadm.py
+1
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
We could refine further this concept if we like it. The idea is to be lightweight as possible while carrying all the information we need.
I agree about keeping global metadata in that way, except ideally using docstrings for better readability (e.g. interactive help()). To better reflect the needs for granular deprecation and based on our private discussion on this subject, I'd like to see at least @deprecated(target_removal_version) decorator. On top of that, we need properly documented public API and deprecation process.
Using standard way such as __author__ is better than docstrings, see for example the output of help(module) Nir
On 29/03/16 20:36 +0300, Nir Soffer wrote:
On Tue, Mar 29, 2016 at 7:44 PM, Martin Polednik <mpolednik@redhat.com> wrote:
On 29/03/16 11:12 -0400, Francesco Romani wrote:
Hi,
in the last Vdsm developer call we agreed to promote a few modules in the common repository. The common repository provides the additional guarantees over regular modules in lib/vdsm/
- stable API - (thus) safe to use across verticals
the planned moves are:
lib/vdsm/schedule.py -> lib/vdsm/common/schedule.py lib/vdsm/periodic.py -> lib/vdsm/common/periodic.py lib/vdsm/virt/api.py -> lib/vdsm/common/api.py
Question is if those modules should go under common/ or under another subdirectory, maybe infra?
Hi.
I agree that some modules should be kept (and advertised) as stable.
The name 'common' is more suited for such package as 'infra' is very ovirt specific and could hurt readability and navigation of the project. It does make sense to structure the project as follows:
lib/vdsm/common lib/vdsm/virt/common lib/vdsm/storage/common lib/vdsm/network/common
This creates a dependency between the packages. The goal is to have packages that do not depend on each other except the common package, which is library code used by other parts.
True, forgot about that. We fortunately still have possibility of further sharding 'common' if that need arises.
Taking extra care taken to avoid 'common junkyard' situation and making sure that the scope of common corresponds to it's package.
List of modules that I consider public (mostly author/main contributor of these lately):
lib/vdsm/cpuarch.py lib/vdsm/cpuinfo.py lib/vdsm/hostdev.py lib/vdsm/machinetype.py lib/vdsm/numa.py lib/vdsm/osinfo.py lib/vdsm/udevadm.py
+1
Lastly, i have a proposal about better handling of those modules.
First, the mere fact a module is placed under lib/vdsm/common provides the extra guarantees I mentioned. But should we added more annotations?
for example something like
__API__ = {}
near the top of the module
if this attribute exist, then the module is safe to use across verticals, has stable API and so forth (this is _in addition_ to the common/ package, not as replacement).
Like:
__API__ = { "introduced-in": "4.14.0", "deprecated-from": "4.18.0", "removed-at": "4.20.0", "contact": "fromani@redhat.com" }
We could refine further this concept if we like it. The idea is to be lightweight as possible while carrying all the information we need.
I agree about keeping global metadata in that way, except ideally using docstrings for better readability (e.g. interactive help()). To better reflect the needs for granular deprecation and based on our private discussion on this subject, I'd like to see at least @deprecated(target_removal_version) decorator. On top of that, we need properly documented public API and deprecation process.
Using standard way such as __author__ is better than docstrings, see for example the output of help(module)
Partially agreed. Having proper docstrings wouldn't hurt either. I frequently use repl w/ vdsm now that it is in lib, this would help myself - am I the only one? I would still like to see some kind of proper deprecation in private APIs. It might as well serve as a testing ground for deprecation in public API.
Nir _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (3)
-
Francesco Romani -
Martin Polednik -
Nir Soffer