'make distcheck' issues in VDSM project (regarding BZ#1297808)
Hi, I've been doing some work around making 'make distcheck' and VPATH builds successful for VDSM. Some patches have already been merged, but there is one that I've abandoned because it touches a bigger issue. When doing a VPATH build, some python sources are build-time generated from '.py.in' files (at this moment: 'lib/vdsm/common/constants.py.in', 'lib/vdsm/common/config.py.in', 'lib/vdsm/common/dsaversion.py.in', '/lib/sos/vdsm.py.in'). Suppose we do a VPATH build with: cd ${SRCROOT} && \ ./autogen.sh && \ make distclean && \ mkdir -p /tmp/ovirt-build && \ cd /tmp/ovirt-build && \ ${SRCROOT}/configure && \ make The build-generated '.py' files will land in '/tmp/ovirt-build/lib/...'. As a result, even though the build-generated python modules and the source ones refer to each other, they can't see each other. After some trial and error here are my observations: - we can't generate resulting '.py' files in $(srcdir) - 'make distcheck' purposely sets $(srcdir) to be read-only - we can't just simply add $(builddir) to PYTHONPATH - there are no '__init__.py' files in $(builddir), so i.e. 'import vdsm.common.config' will fail - we can't create dummy '__init__.py' files in $(builddir), because that would break in-tree-builds (we would have to clean regular, git-maintained files) In conclusion, I think we would need to implement a custom python module loader to make VPATH/distcheck successful. It would need to be parametrized with $(builddir)'s path, and would try to resolve unsuccessful imports by finding build-time generated files inside. However, after looking at other oVirt projects (i.e. 'ioprocess', 'ovirt-iso-uploader', 'ovirt-node-ng', 'ovirt-register'), I can see that most of them fail 'make distcheck'. They have even more severe 'makefile'-issues than importing build-time generated python files. So the question is - do we really need 'make distcheck' for VDSM to suceed? Marcin
Il giorno mer 26 set 2018 alle ore 15:33 Marcin Sobczyk <msobczyk@redhat.com> ha scritto:
Hi,
I've been doing some work around making 'make distcheck' and VPATH builds successful for VDSM. Some patches have already been merged, but there is one that I've abandoned because it touches a bigger issue.
When doing a VPATH build, some python sources are build-time generated from '.py.in' files (at this moment: 'lib/vdsm/common/constants.py.in', 'lib/vdsm/common/config.py.in', 'lib/vdsm/common/dsaversion.py.in', '/lib/sos/vdsm.py.in').
Suppose we do a VPATH build with:
cd ${SRCROOT} && \ ./autogen.sh && \ make distclean && \ mkdir -p /tmp/ovirt-build && \ cd /tmp/ovirt-build && \ ${SRCROOT}/configure && \ make
The build-generated '.py' files will land in '/tmp/ovirt-build/lib/...'. As a result, even though the build-generated python modules and the source ones refer to each other, they can't see each other.
After some trial and error here are my observations: - we can't generate resulting '.py' files in $(srcdir) - 'make distcheck' purposely sets $(srcdir) to be read-only - we can't just simply add $(builddir) to PYTHONPATH - there are no '__init__.py' files in $(builddir), so i.e. 'import vdsm.common.config' will fail - we can't create dummy '__init__.py' files in $(builddir), because that would break in-tree-builds (we would have to clean regular, git-maintained files)
In conclusion, I think we would need to implement a custom python module loader to make VPATH/distcheck successful. It would need to be parametrized with $(builddir)'s path, and would try to resolve unsuccessful imports by finding build-time generated files inside.
However, after looking at other oVirt projects (i.e. 'ioprocess', 'ovirt-iso-uploader', 'ovirt-node-ng', 'ovirt-register'), I can see that most of them fail 'make distcheck'. They have even more severe 'makefile'-issues than importing build-time generated python files.
So the question is - do we really need 'make distcheck' for VDSM to suceed?
Well, if it's not succeeding we have a bug :-) Now, that bug may be just a noise or a big issue: if it's just a noise, we have more urgent stuff to fix and this can stay in the backlog.
Marcin
-- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://red.ht/sig> <https://www.redhat.com/en/events/red-hat-open-source-day-italia?sc_cid=701f2000000RgRyAAK>
participants (2)
-
Marcin Sobczyk -
Sandro Bonazzola