[ovirt-devel] Re: Running Vdsm on Python 3

Hi, there have been efforts by different Vdsm developers to port Vdsm to Python 3. I've joined the efforts recently and I focused on the aim to make Vdsm running (not necessarily working correctly initially) on Python 3, in order to better understand all the related problems and to test Python 3 compatibility (among other) on a running Vdsm instance. I can currently start Vdsm on Python 3, although it can't communicate with clients yet (such as Engine), see below. FWIW, here is what I had to do to make Vdsm at least installable and startable with Python 3. I use Fedora 28 as the platform. The primary reason is that ovirt.org provides for Fedora 28 all the non-Vdsm oVirt packages needed to run Vdsm on the host. Direct host installation doesn't work for me yet with Python 3. It might be a nice next goal but there is an added complexity in requiring proper support in all the related oVirt packages, not just Vdsm. It's better to have a basically working Vdsm first. So I had installed the host with Python 2 Vdsm and then upgraded Vdsm to Python 3 version. One non-Vdsm Python package required by Vdsm and not yet ported to Python 3 is sanlock-python. There is an ongoing work on Python 3 port at https://pagure.io/fork/mcsontos/sanlock/commits/dev-mcsontos-python3. I didn't know about it so I used my own hack that made sanlock Python module at least importable on Python 3. Let's discuss it on sanlock-devel. Regarding Vdsm itself, there are several areas to fix. First, there are compatibility problems in Python code. I started making some fixes in a patch series starting with https://gerrit.ovirt.org/97019. Please note there are other Python 3 patches by other Vdsm developers. I hope that once we branch out 4.3, we can start merging the patches to master quickly and better coordinate our efforts. Then there are other changes to Vdsm needed, which I don't have patches for (just my ugly private hacks):

- All the /usr/bin/python2 shebangs must be replaced with /usr/bin/python3. We should find a good way to solve and automated that, perhaps as part of the installation process depending on Python version selected. - configure.ac must be adjusted: + AM_PATH_PYTHON([3.6]) + AC_PATH_PROG([PYTHON_CONFIG], [python3-config]) + AX_PYTHON_MODULE([six], [fatal], python3) It should be easy to make a patch for that. - The following changes in vdsm.spec.in are needed: + Dependencies on Python packages must be adjusted. + python3_sitelib must be used instead of python_sitelib for Python 3. + Byte compiled files are placed into __pycache__ subdirectories by Python 3 and they must be installed. It would be nice to have patches for that so that we can build Python 3 rpm's easily. That's it at the moment. Most compatibility problems in Python code are easy to fix once they are identified. One important exception is bytes versus strings -- that's going to be delicate. The bytes versus strings problem is very present in communication code -- such as vdsm/rpc/http.py, vdsm/sslutils.py, etc. While strings constants are typically used in the code, the communication means work with bytes. This mismatch prevents Vdsm from communicating with the clients, so it would be very useful if someone could look at it and fix the code (one of the harder parts would probably be to decide where to keep bytes and where to already convert them to strings). HTH. As more people are involved in porting Vdsm to Python 3, it might be useful to coordinate all the efforts here, to avoid duplicate work, to share findings and to speed up the process.