gh-131738: optimize builtin any/all/tuple calls with a generator expression arg

[iritkatriel]

This PR implements the optimization described in faster-cpython/ideas#545.

• Issue: optimize any/all/tuple calls with a generator expression arg #131738

[TeamSpen210]

I don't think LOAD_COMMON_CONST is safe right now, since users could have messed with the builtins module. Probably better to stash the funcs in interp->callable_cache like other builtins used for optimisation?

[brandtbucher]

Very cool! Any plans to do this for set or list?

dict/max/min/sum are trickier, but may be worth exploring too.

[bedevere-app]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

[brandtbucher]

This should fix the JIT builds:

diff --git a/Tools/jit/_targets.py b/Tools/jit/_targets.py
index aa2b56abf44..73a4210eee0 100644
--- a/Tools/jit/_targets.py
+++ b/Tools/jit/_targets.py
@@ -522,7 +522,14 @@ def get_target(host: str) -> _COFF | _ELF | _MachO:
         args = ["-fms-runtime-lib=dll"]
         target = _COFF(host, args=args)
     elif re.fullmatch(r"x86_64-.*-linux-gnu", host):
-        args = ["-fno-pic", "-mcmodel=medium", "-mlarge-data-threshold=0"]
+        args = [
+            # Jump tables generate R_X86_64_32S relocations, which assume a
+            # signed 32-bit address space:
+            "-fno-jump-tables",
+            "-fno-pic",
+            "-mcmodel=medium",
+            "-mlarge-data-threshold=0",
+        ]
         target = _ELF(host, args=args)
     else:
         raise ValueError(host)

[markshannon]

The overall approach looks good.

Do you have stats for this PR?
If there is enough of an increase in the execution counts for CALL_INTRINSIC_1 then we should move CALL_INTRINSIC_1 INTRINSIC_LIST_TO_TUPLE into a new LIST_TO_TUPLE instruction.

[rhettinger]

dict/max/min/sum are trickier, but may be worth exploring too.

The latter three would be nice wins.

[iritkatriel]

Perf results are here: https://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20250326-3.14.0a6%2B-50dd66b/README.md

[iritkatriel]

Do you have stats for this PR? If there is enough of an increase in the execution counts for CALL_INTRINSIC_1 then we should move CALL_INTRINSIC_1 INTRINSIC_LIST_TO_TUPLE into a new LIST_TO_TUPLE instruction.

[iritkatriel]

I have made the requested changes; please review again.

[bedevere-app]

Thanks for making the requested changes!

@brandtbucher: please review the changes made to this pull request.

[brandtbucher]

Looks good. You can revert the JIT change, it's not needed anymore.

(Also I'm running JIT benchmarks just because I'm curious, but no need to block on those results at all.)

[brandtbucher]

Is this needed?

[iritkatriel]

Yes, I had to remove this include from another header so it needs to be included when needed.

[brandtbucher]

(Also, I think you need to make regen-cases.)