[VDSM] Handling of scripts without a .py suffix
Hi all, We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with python3, and we have similar (but working) whitelist for pyflakes and pep8. To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix. With these changes, we can use the various checking commands on the entire source tree. For example, these commands check the entire tree: PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes . Thoughts? [1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363 Nir
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with python3, and we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
On Sun, May 29, 2016 at 12:36 PM, Dan Kenigsberg <danken@redhat.com> wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote: python3, and
we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
we can leave the py suffix. it doesn't really matter
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
I didn't comment yet because I don't like the idea behind "helpers" folder in vdsm tree, and I don't have any better offers except "scripts". we can also migrate those scripts to python.. not so sure how much value we can get from it if any _______________________________________________
Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *Yaniv Bronhaim.*
On Sun, May 29, 2016 at 1:02 PM, Yaniv Bronheim <ybronhei@redhat.com> wrote:
On Sun, May 29, 2016 at 12:36 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with python3, and we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
we can leave the py suffix. it doesn't really matter
I think it does. If I rewrite fc-scan in another language, I don't want to change the code calling. Executables should not have a suffix. (We also need to fix the protect shell scripts, remove the .sh)
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
I didn't comment yet because I don't like the idea behind "helpers" folder in vdsm tree, and I don't have any better offers except "scripts".
scripts is not correct, these are executables that can be implemented in any language. Their purpose is to help vdsm perform several tasks that can not be done from the vdsm process (e.g. fc-scan).
we can also migrate those scripts to python.. not so sure how much value we can get from it if any
We are not going to touch the protect scripts, these are legacy and delicate, we are waiting for the moment that we can delete them :-) Nir
On Sun, May 29, 2016 at 12:36 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with python3, and we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
This is a separate topic. The helpers do not belong in the library - we should keep libv/vdsm/xxx with only the code that is needed for the xxx package. Helpers are external programs that should have access only to public vdsm apis, so they don't need to and should not have access to other files inside lib/vdsm/xxx. This also make the source easier to understand, the structure is closer to the final structure after installation. So we can have: helpers/storage helpers/virt ... But I don't see any value in this separation, we have only about 10 helpers. Also each directory we add adds overhead of more useless autotools files to maintain. Look how many makefiles we got rid by moving all the tests to one directory. Nir
On Sun, May 29, 2016 at 02:53:41PM +0300, Nir Soffer wrote:
On Sun, May 29, 2016 at 12:36 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with python3, and we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
This is a separate topic.
The helpers do not belong in the library - we should keep libv/vdsm/xxx with only the code that is needed for the xxx package. Helpers are external programs that should have access only to public vdsm apis, so they don't need to and should not have access to other files inside lib/vdsm/xxx.
This also make the source easier to understand, the structure is closer to the final structure after installation.
So we can have:
helpers/storage helpers/virt ...
But I don't see any value in this separation, we have only about 10 helpers.
Also each directory we add adds overhead of more useless autotools files to maintain. Look how many makefiles we got rid by moving all the tests to one directory.
Ok, I'm convinced.
On Sun, May 29, 2016 at 3:31 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sun, May 29, 2016 at 12:36 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Sat, May 28, 2016 at 03:16:10PM +0300, Nir Soffer wrote:
Hi all,
We have several scripts spread in the source, typically installed in /usr/libexec/vdsm. We had a useless WHITELIST[1], trying to compile these scripts with
On Sun, May 29, 2016 at 02:53:41PM +0300, Nir Soffer wrote: python3, and
we have similar (but working) whitelist for pyflakes and pep8.
To simplify the various checks, I think we need to to do this: 1. Keep .py suffix for all python files 2. Move all scripts to helpers/ ([2] handles storage scripts) 3. During installation, strip the .py suffix.
With these changes, we can use the various checking commands on the entire source tree.
For example, these commands check the entire tree:
PYTHONDONTWRITEBYTECODE=1 python3 -m compileall -f -x '(\.tox/|\.git/)' . pep8 . pyflakes .
Thoughts?
[1] https://gerrit.ovirt.org/58204 [2] https://gerrit.ovirt.org/57363
Sounds good, though I'd love to keep the separation of scripts into their natuaral vertical. Keep storage understand storage, etc. Why are you piling them into one source directory?
This is a separate topic.
The helpers do not belong in the library - we should keep libv/vdsm/xxx with only the code that is needed for the xxx package. Helpers are external programs that should have access only to public vdsm apis, so they don't need to and should not have access to other files inside lib/vdsm/xxx.
This also make the source easier to understand, the structure is closer to the final structure after installation.
So we can have:
helpers/storage helpers/virt ...
But I don't see any value in this separation, we have only about 10 helpers.
Also each directory we add adds overhead of more useless autotools files to maintain. Look how many makefiles we got rid by moving all the tests to one directory.
Ok, I'm convinced.
iiuc you plan to change the source tree only, after installation those executables will stay under /usr/libexec/vdsm . we have helper folder since https://gerrit.ovirt.org/55797 , now we need to expect verticals there? ok by me.
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *Yaniv Bronhaim.*
participants (3)
-
Dan Kenigsberg -
Nir Soffer -
Yaniv Bronheim