Build failure with Python 3.11

[nanonyme]

Describe the bug

jack2 tells Python to use mode rUb which Python 3.11 rejects as invalid. As a result this project cannot be built. The invalid mode is constructed from here https://github.com/jackaudio/jack2/blob/v1.9.21/waflib/Context.py#L665

Environment

• JACK Version: 1.9.21
• Operating System: Linux
• Installation: Source

Steps To Reproduce

Install a distro with Python 3.11, eg Fedora 37 beta and try to build jack2 with waflib.

Expected vs. actual behavior

Expected behaviour is jack2 builds, actual behaviour is Python stacktrace on invalid mode.

        Waf: The wscript in '/buildstream/freedesktop-sdk/components/jack.bst' is unreadable
        Traceback (most recent call last):
          File "/buildstream/freedesktop-sdk/components/jack.bst/waflib/Scripting.py", line 141, in waf_entry_point
            set_main_module(wscript)
          File "/buildstream/freedesktop-sdk/components/jack.bst/waflib/Scripting.py", line 191, in set_main_module
            Context.g_module = Context.load_module(file_path)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/buildstream/freedesktop-sdk/components/jack.bst/waflib/Context.py", line 665, in load_module
            code = Utils.readf(path, m='rU', encoding=encoding)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/buildstream/freedesktop-sdk/components/jack.bst/waflib/Utils.py", line 231, in readf
            with open(fname, m) as f:
                 ^^^^^^^^^^^^^^
        ValueError: invalid mode: 'rUb'

[nedko]

This issue was addressed by waf upstream 3 years ago: https://gitlab.com/ita1024/waf/-/commit/68997828c850ce7fb30b73b4adfde35053e539d1

Upgrading waf to latest should fix the py3.11 incompatibility.

See also #884

[nanonyme]

Do you mean no one has updated waf for last three years?

[falkTX]

it is not something projects are expected to do to be honest..
if one uses others like cmake and meson, there is pretty much a guarantee that once setup things will keep working, perhaps only breaking for toolchain updates but that would be something to fix in code rather than the build system.

[nedko]

Is jackaudio/jack2 going to be switched away from waf?

[falkTX]

eventually switching to meson yes

[ionenwks]

eventually switching to meson yes

Any progress regarding this? fwiw the bundled waf is yet again broken if try to use python 3.12 (3.12 still in beta atm so not really an issue yet), given 3.12 removed the deprecated imp module resulting in:

    from waflib import Logs, Utils, Context, Errors
  File "/tmp/portage/media-sound/jack2-1.9.22/work/jack2-1.9.22-abi_x86_32.x86/waflib/Context.py", line 9, in <module>
    import os, re, imp, sys
ModuleNotFoundError: No module named 'imp'

Alternatively may be an easy fix in waf like did with 3.11, haven't really looked.

[mohd-akram]

This is fixed in version 1.9.22.

[Schamschula]

eventually switching to meson yes

Any progress regarding this? fwiw the bundled waf is yet again broken if try to use python 3.12 (3.12 still in beta atm so not really an issue yet), given 3.12 removed the deprecated imp module resulting in:

    from waflib import Logs, Utils, Context, Errors
  File "/tmp/portage/media-sound/jack2-1.9.22/work/jack2-1.9.22-abi_x86_32.x86/waflib/Context.py", line 9, in <module>
    import os, re, imp, sys
ModuleNotFoundError: No module named 'imp'

Alternatively may be an easy fix in waf like did with 3.11, haven't really looked.

I want to exclusively use Python 3.12. So I looked into this. All it took was a small patch to waflib/Context.py

--- waflib/Context.py.orig	2023-02-02 05:04:10
+++ waflib/Context.py	2024-06-23 15:42:02
@@ -6,7 +6,7 @@
 Classes and functions enabling the command system
 """
 
-import os, re, imp, sys
+import os, re, sys, types
 from waflib import Utils, Errors, Logs
 import waflib.Node
 
@@ -660,7 +660,7 @@
 	except KeyError:
 		pass
 
-	module = imp.new_module(WSCRIPT_FILE)
+	module = types.ModuleType(WSCRIPT_FILE)
 	try:
 		code = Utils.readf(path, m='r', encoding=encoding)
 	except EnvironmentError:

See: https://docs.python.org/3/library/types.html#types.ModuleType

[ionenwks]

@Schamschula for the record, the 3.12 issues with imp are already fixed in git (by updating waf), but we're still waiting for release to include the change (issue #974). Albeit a short patch may be useful for people wanting to backport something smaller, the upstream fix is to update waf. In Gentoo we've backported the waf bump though (applies as-is).

[Schamschula]

@Schamschula for the record, the 3.12 issues with imp are already fixed in git (by updating waf), but we're still waiting for release to include the change (issue #974). Albeit a short patch may be useful for people wanting to backport something smaller, the upstream fix is to update waf. In Gentoo we've backported the waf bump though (applies as-is).

No problem! Looking forward to the update! Jack was blocking other ports, so I needed a workaround.