[ovirt-devel] Re: Running Vdsm on Python 3

On Thu, Feb 7, 2019 at 11:04 PM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: > > Hi, there have been efforts by different Vdsm developers to port Vdsm to > Python 3. I've joined the efforts recently and I focused on the aim to > make Vdsm running (not necessarily working correctly initially) on > Python 3, in order to better understand all the related problems and to > test Python 3 compatibility (among other) on a running Vdsm instance. > > I can currently start Vdsm on Python 3, although it can't communicate > with clients yet (such as Engine), see below. FWIW, here is what I had > to do to make Vdsm at least installable and startable with Python 3. > > I use Fedora 28 as the platform. The primary reason is that ovirt.org > provides for Fedora 28 all the non-Vdsm oVirt packages needed to run > Vdsm on the host. > > Direct host installation doesn't work for me yet with Python 3. It > might be a nice next goal but there is an added complexity in requiring > proper support in all the related oVirt packages, not just Vdsm. It's > better to have a basically working Vdsm first. So I had installed the > host with Python 2 Vdsm and then upgraded Vdsm to Python 3 version. > > One non-Vdsm Python package required by Vdsm and not yet ported to > Python 3 is sanlock-python. There is an ongoing work on Python 3 port > at https://pagure.io/fork/mcsontos/sanlock/commits/dev-mcsontos-python3. > I didn't know about it so I used my own hack that made sanlock Python > module at least importable on Python 3. Let's discuss it on > sanlock-devel. > > Regarding Vdsm itself, there are several areas to fix. First, there are > compatibility problems in Python code. I started making some fixes in a > patch series starting with https://gerrit.ovirt.org/97019. Please note > there are other Python 3 patches by other Vdsm developers. I hope that > once we branch out 4.3, we can start merging the patches to master > quickly and better coordinate our efforts. > > Then there are other changes to Vdsm needed, which I don't have patches > for (just my ugly private hacks): I believe that +Gal Zaidman has few patches for these in his drawer. He has postponed his work due to 4.3 stabilitzation.

> - All the /usr/bin/python2 shebangs must be replaced with > /usr/bin/python3. We should find a good way to solve and automated > that, perhaps as part of the installation process depending on Python > version selected. > > - configure.ac must be adjusted: > > + AM_PATH_PYTHON([3.6]) > + AC_PATH_PROG([PYTHON_CONFIG], [python3-config]) > + AX_PYTHON_MODULE([six], [fatal], python3) > > It should be easy to make a patch for that. > > - The following changes in vdsm.spec.in are needed: > > + Dependencies on Python packages must be adjusted. > + python3_sitelib must be used instead of python_sitelib for Python 3. > + Byte compiled files are placed into __pycache__ subdirectories by > Python 3 and they must be installed. > > It would be nice to have patches for that so that we can build > Python 3 rpm's easily. > > That's it at the moment. Most compatibility problems in Python code are > easy to fix once they are identified. One important exception is bytes > versus strings -- that's going to be delicate. > > The bytes versus strings problem is very present in communication > code -- such as vdsm/rpc/http.py, vdsm/sslutils.py, etc. While strings > constants are typically used in the code, the communication means work > with bytes. This mismatch prevents Vdsm from communicating with the > clients, so it would be very useful if someone could look at it and fix > the code (one of the harder parts would probably be to decide where to > keep bytes and where to already convert them to strings). > > HTH. As more people are involved in porting Vdsm to Python 3, it might > be useful to coordinate all the efforts here, to avoid duplicate work, > to share findings and to speed up the process. Indeed. We may also try to revive Trello usage. Would you add relevant tasks to https://trello.com/b/tHeO1BFW/py3-vdsm ?