Combining Paramspec Union

[hmc-cs-mdrissi]

Summary

When using paramspecs is union present with P.args/P.kwargs intended to be legal or not?

from typing import Callable, Sequence
from typing_extensions import ParamSpec
P = ParamSpec('P')

def foo(func: Callable[P, object], *args: P.args | Sequence[object], **kwargs: P.kwargs | Sequence[object]) -> None:
  ...

# Example toy cases,
def add(x: int, y: int) -> int:
  ...

foo(add, 3, 2) # Good
foo(add, ["hi"], 2) # Good
foo(add, x=3, y=["hi"]) # Good
foo(add, "hi", 2) # Error string passed expected either int or Sequence[object]

I looked over pep 612, but it does not specify how unions interact with P.args/P.kwargs. Here I'm interpreting union as each argument either matches Sequence[object] or is compatible P.args (original arg type). My primary use case is for similar problems as this prior question. If this is supported then function mentioned there could be hinted as,

def remote(func: Callable[P, R], *args: P.args | ObjectRef[object], **kwargs: P.kwargs | ObjectRef[object]) -> ObjectRef[R]:
  ...

Still not fully precise as real intent is closer to ObjectRef[X] where X matches original argument type, but it would get much closer without requiring Map operator.

edit: Pondering this it feels a bit messy semantic. Since removing Paramspec and just doing

def foo(func, *args: Sequence[object], **kwargs: Sequence[object]):
  ...

foo(add, [1], [2], [3]) 

would be legal although number of arguments is now a mismatch with number add had. So my guess is unions don't fit together well with P.args and type hinting function like remote remains hard.

[erictraut]

No, this isn't allowed. P.args and P.kwargs must always be used in conjunction with each other within a signature, and combining them with other types in a union violates this. You could use an overload where the first signature uses a ParamSpec and the second does not.

[whyh]

overloaded function should still have the both types in the final signature, here is an example, where this is not possible

OriginalReturn = Any
OriginalParams = ParamSpec("OriginalParams")
OriginalFunction = Callable[[OriginalParams.args, OriginalParams.kwargs], OriginalReturn]

@overload
def log_exceptions(function: OriginalFunction) -> OriginalFunction:
    ...


@overload
def log_exceptions(*exceptions: Type[BaseException]) -> Callable[[OriginalFunction], OriginalFunction]:
    ...


def log_exceptions(
    function_or_exception: Type[BaseException] | OriginalFunction = BaseException, *exceptions: Type[BaseException]
) -> Callable[[OriginalFunction], OriginalFunction] | OriginalFunction
    pass

This can be somewhat mitigated by putting OriginalFunction in quotes

I might be doing something very wrong here, please let me know it it's the case

[erictraut]

@whyh, yes, you're doing something very wrong here. :)

You cannot use P.args and P.kwargs within a Callable type argument. The P.args and P.kwargs forms are allowed only as type annotations for *args and **kwargs parameters within a def statement. Refer to PEP 612 for details and examples.

You also seem to be attempting to define a generic type alias called OriginalFunction that is parameterized by a single ParamSpec type parameter. But then when you make use of that generic type alias, you don't provide a type argument corresponding to this type parameter. Type checkers will therefore assume you mean OriginalFunction[...] (since ... is the equivalent of Any for a ParamSpec).

I assume that you're trying to figure out how to properly annotate a decorator function. This would look something like this:

from typing import Callable, ParamSpec, TypeVar, overload

P1 = ParamSpec("P1")
R1 = TypeVar("R1")
P2 = ParamSpec("P2")
R2 = TypeVar("R2")

@overload
def log_exceptions(function: Callable[P1, R1], /) -> Callable[P1, R1]:
    ...

@overload
def log_exceptions(
    first_exception: type[BaseException] = BaseException,
    /,
    *additional_exceptions: type[BaseException],
) -> Callable[[Callable[P2, R2]], Callable[P2, R2]]:
    ...

def log_exceptions(
    function_or_exception: type[BaseException] | Callable[P1, R1] = BaseException,
    /,
    *additional_exceptions: type[BaseException],
) -> Callable[[Callable[P2, R2]], Callable[P2, R2]] | Callable[P1, R1]:
    ...