Skip to content

Compatibility issues with weakrefs #1102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BobTheBuidler opened this issue May 8, 2025 · 1 comment · May be fixed by python/mypy#19056
Open

Compatibility issues with weakrefs #1102

BobTheBuidler opened this issue May 8, 2025 · 1 comment · May be fixed by python/mypy#19056

Comments

@BobTheBuidler
Copy link

BobTheBuidler commented May 8, 2025

I tried to use a compiled class with weakrefs and had mixed results. Here are my findings:

Not working:

class MyFailingClass0:
    pass

obj = MyFailingClass0()
ref = weakrefs.ref(obj)

@final
class MyFailingClass1:
    pass

obj = MyFailingClass1()
ref = weakrefs.ref(obj)

I then thought about my experience with cython and tried these, also not working:

class MyFailingClass2:
    def __init__(self):
        self.__weakref__: Final[Optional[ReferenceType]] = None

obj = MyFailingClass2()
ref = weakrefs.ref(obj)

@final
class MyFailingClass3:
    def __init__(self):
        self.__weakref__: Final[Optional[ReferenceType]] = None

obj = MyFailingClass3()
ref = weakrefs.ref(obj)

Next I tried giving my class a dict attribute manually, also not working:

class MyFailingClass4:
    def __init__(self):
        self.__dict__: Final[Dict[str, Any]] = {}

obj = MyFailingClass4()
ref = weakrefs.ref(obj)

@final
class MyFailingClass5:
    def __init__(self):
        self.__dict__: Final[Dict[str, Any]] = {}

obj = MyFailingClass5()
ref = weakrefs.ref(obj)

I think the above results show us a few things are not working as they probably should be:

  1. a user should be able to add a dict attribute to a compiled class, either by defining it in the class init or with mypyc_attr(dict=True) similar to how msgspec handles it, or both. Based on what I saw here https://github.com/python/mypy/blob/501a07b45af8e44eda665e53526fc590dc5a014e/mypyc/codegen/emitclass.py#L285 I believe if this is implemented then weakrefs would also work for those classes. It might also be good to autodetect uses of functools.cached_property and give those classes a dict attr as well.
  2. a user should be able to add a weakref attribute to a compiled class, by one or more of the following: defining it in the class init, defining it in slots, with mypyc_attr(supports_weakref=True)

However! I was able to find a pattern that succeeds with weakrefs currently:

noncompiledfile.py

class WeakrefBase:
    """We need our class to inherit from a python class in order for weakrefs to work."""

compiledfile.py

from noncompiledfile import WeakrefBase

class Weakrefable(WeakrefBase):
    ...

obj = Weakreafable()
ref = weakrefs.ref(obj)

Success!

@BobTheBuidler BobTheBuidler linked a pull request May 8, 2025 that will close this issue
@BobTheBuidler
Copy link
Author

BobTheBuidler commented May 8, 2025

It might also be good to raise a compiler error if weakref.ref or weakref.proxy is called on a compiled class instance that does not support weakrefs.

Currently, the user only learns there is an issue at runtime

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant