[ovirt-devel] python path conflicts issues while running python scripts from vdsm verbs

[Ramesh Nachimuthu]

----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Aravinda" <avishwan at redhat.com>, igoihman at redhat.com
> Cc: "Ramesh Nachimuthu" <rnachimu at redhat.com>, "Nir Soffer" <nsoffer at redhat.com>, "devel" <devel at ovirt.org>, "Piotr
> Kliczewski" <pkliczew at redhat.com>, "Yaniv Bronheim" <ybronhei at redhat.com>, "Sahina Bose" <sabose at redhat.com>
> Sent: Wednesday, November 30, 2016 2:02:43 PM
> Subject: Re: python path conflicts issues while running python scripts from vdsm verbs
> 
> 
> Regardless of the solution you plan to take, let it serve as a reminder
> about moving gluster bits out of /usr/share/vdsm/ and into
> site-packages. I am willing to pay the price of an ugly
> exception-handling in xmlrpc, if this is the cost.

Yes. I think this is the right time to revive the patch https://gerrit.ovirt.org/#/c/57530/ and https://gerrit.ovirt.org/#/c/57524/ to move all the gluster code to site-packages. Let me re-post these packages for review.

Regards,
Ramesh

> 
> On Wed, Nov 30, 2016 at 11:27:06AM +0530, Aravinda wrote:
> > If vdsm setting PYTHONPATH as below
> > 
> >     export PYTHONPATH=/usr/share/vdsm:$PYTHONPATH
> > 
> > change it as
> > 
> >     export PYTHONPATH=$PYTHONPATH:/usr/share/vdsm
> > 
> > With this change, /usr/share/vdsm will not get first preference compared to
> > /usr/lib*/python2.7/site-packages
> > 
> > regards
> > Aravinda
> > 
> > On 11/30/2016 10:25 AM, Ramesh Nachimuthu wrote:
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Nir Soffer" <nsoffer at redhat.com>
> > > > To: "Ramesh Nachimuthu" <rnachimu at redhat.com>
> > > > Cc: "devel" <devel at ovirt.org>, "Dan Kenigsberg" <danken at redhat.com>,
> > > > "Piotr Kliczewski" <pkliczew at redhat.com>, "Yaniv
> > > > Bronheim" <ybronhei at redhat.com>, "Aravinda Vishwanathapura Krishna
> > > > Murthy" <avishwan at redhat.com>, "Sahina Bose"
> > > > <sabose at redhat.com>
> > > > Sent: Tuesday, November 29, 2016 7:20:57 PM
> > > > Subject: Re: python path conflicts issues while running python scripts
> > > > from vdsm verbs
> > > > 
> > > > On Tue, Nov 29, 2016 at 3:26 PM, Ramesh Nachimuthu
> > > > <rnachimu at redhat.com>
> > > > wrote:
> > > > > Hi,
> > > > > 
> > > > >   I am trying to run a python script '/sbin/gluster-eventsapi' in
> > > > >   vdsm verb
> > > > >   which internally imports some python modules from
> > > > >   /usr/lib/python2.7/site-packages/gluster/cliutils/. But it fails
> > > > >   with the
> > > > >   import error. Following error is seen in the supervdsm log.
> > > > > 
> > > > > 
> > > > > MainProcess|Thread-1::DEBUG::2016-11-28
> > > > > 16:54:35,130::commands::93::root::(execCmd) FAILED: <err> =
> > > > > 'Traceback
> > > > > (most recent call last):\n  File "/sbin/gluster-eventsapi", line 25,
> > > > > in
> > > > > <module>\n    from gluster.cliutils.cliutils import (Cmd, execute,
> > > > > node_output_ok, node_output_notok,\nImportError: No module named
> > > > > cliutils.cliutils\n'; <rc> = 1
> > > > > 
> > > > > 
> > > > > I think the import statement "from gluster.cliutils.cliutils import
> > > > > (Cmd,
> > > > > execute, node_output_ok, node_output_notok)" in the python script
> > > > > resolves
> > > > > to '/usr/share/vdsm/gluster' instead of
> > > > > /usr/lib/python2.7/site-packages/gluster/cliutils.
> > > > > 
> > > > > I see the following in python sys.path while executing a python
> > > > > script from
> > > > > vdsm.
> > > > > 
> > > > > ['/usr/libexec/glusterfs', '/usr/share/vdsm',
> > > > > '/usr/lib64/python27.zip',
> > > > > '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2',
> > > > > '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old',
> > > > > '/usr/lib64/python2.7/lib-dynload',
> > > > > '/usr/lib64/python2.7/site-packages',
> > > > > '/usr/lib64/python2.7/site-packages/gtk-2.0',
> > > > > '/usr/lib/python2.7/site-packages']
> > > > > 
> > > > > Looks like '/usr/share/vdsm' take precedence over
> > > > > '/usr/lib64/python2.7/site-packages'.
> > > > > 
> > > > > Can someone suggests a way to fix this issue?
> > > > > 
> > > > > Note: '/sbin/gluster-eventsapi' works perfectly while running
> > > > > directly from
> > > > > CLI.
> > > > > 
> > > > > Related vdsm patch:
> > > > > https://gerrit.ovirt.org/#/c/67168/2/vdsm/gluster/events.py
> > > > If an import fails because /usr/share/vdsm/foo hides a package in
> > > > /usr/lib/python2.7/site-packages/vdsm it means that you have a wrong
> > > > import, you need to import anything like this:
> > > > 
> > > >      from vdsm.foo import bar
> > > > 
> > > > So names in /usr/share/vdsm/* cannot hide names in vdsm package.
> > > > 
> > > Here its a different case. Some external python script (which I am trying
> > > to run from vdsm verb) is trying to import some gluster package from
> > > /usr/lib/python2.7/site-packages/gluster. But it is conflicting with
> > > gluster modules under /usr/share/vdsm/gluster
> > > 
> > > Python Script which I am trying to run:
> > > https://github.com/gluster/glusterfs/blob/master/events/src/peer_eventsapi.py
> > > 
> > > Regards,
> > > Ramesh
>