python3 transition

The python3 transition within WMCore isn't really a migration to python3, but it's meant to be a modernization of our code such that it's compatible with python 2.7 and python 3 (latest stable release being 3.8.x at the moment). It's unclear whether python 2.6 would have to be supported as well (especially for the WMRuntime package).

Many of the CMS Computing services are maintained and built by our own group, as well as many of their dependencies. The CMS Computing software stack is currently maintained in this repository/branch: https://github.com/cms-sw/cmsdist/tree/comp_gcc630 where we also build many of the python libraries (either for python2 or python3). Thus, during this python migration, there will be the need to also build new (python) spec files for the required dependencies; and/or to update those that are out-dated or inconsistent between py2 and py3. We use this model such that we have full control of all the dependencies shipped with our CMS software, including any possible patches needed.

This link https://docs.python.org/3/howto/pyporting.html has a lot of good stuff on all the differences between python 2 and 3. The way we have planned this migration considers passing our code through python-futurize http://python-future.org/automatic_conversion.html . Developers should consider doing this now when changing existing code and validating unit tests.

Py2 to Py3 transition work for WMCore

Work of Summer student on py2 to py3 transition is summarized

• wiki page describing all performed steps
• twiki page summarizing futurize steps for different use-cases

Individual notes

items(), keys(), and values()

Python2 uses these keywords to and creates lists of them. This can take a lot of memory. Python3 has the same syntax, but they are now iterators. In Python2, iteritems(), iterkeys(), and itervalues() behave the same as the Python3 versions. There is a "problem" with the futurist fixer for these issues in that it

• Converts python2 uses of items() to list(items()) - not a problem, this is just explicit
• Converts iteritems() to items()
• This is OK for python3, but on python2 possibly alters the performance of the code
• And if you convert this again you now end up with list(items()) in your python3 code altering the performance under Python3 too

Eric's proposal is to use the futurize fixer, but discard all changes that change iteritems() etc to the python 3 versions and use that as the python2 code. We would run it a second time to create a dedicated python3 version. We could also take this opportunity to review our uses of items(), etc in python2 since in most cases we could be using the iterator versions. The most common case where we can't do this is in doing something like len(items()).