-
-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Show __init__ exports #39
Comments
BTW, there are official rules for what's considered a re-exporting of symbols, which is followed by |
This applies not only to |
There was also some parallel discussion in gitter. @pawamoy said:
So I have mainly 3 reasons why I think this should be supported despite your reasons:
About other linters not supporting it, yes, I agree it is annoying, but because of the reasons above, I think they should be, and I'm willing to open issues in |
Thanks for all the info! Just to play devil's advocate a bit (don't read it as "I really don't want to support
Emphasis on "can". To me it sounds like type checkers have a choice in how they consider an object to be public/exported or not. If they want, they can follow these few rules described in the document, but apparently they don't have to. This explicitly says it's for stub files. The only mention of non-stub files is:
So to me, this again is not an authoritative document to describe how to determine whether an object is public/exported or not. I don't think such a document exist yet. Anyway. Even if we don't have specs, we have conventions, and About your third point, there is a way: if you define
# module.py
__all__ = ["X", "Y", "Z"]
... # __init__.py
# __all__ was by the way specifically made for wildcard imports
from module import *
# we could also avoid using the wildcard import,
# but I understand it's less convenient: from module import X, Y, Z
# import module's __all__!
from module import __all__ as _module_all
# use it to extend your parent __all__
__all__ = ["A", "B", "C", *_module_all]
# other operations are supported like:
# __all__ += _module_all About the error-proneness of having to remember to explicitly import and add to Anyway, I think wildcard imports are OK when combined with |
One more argument in favor of using If you use wildcard imports without
import third_party # by convention, not exported
class A: # by convention, exported
... # __init__.py
# by convention, only supposed to export A,
# but at runtime, third_party is exposed too!
from module import * |
🎉
Yeah, I thought of that, but that just means now I need to move all the duplication to the submodules, not that I get rid of the duplication. I can already mark symbols private in submodules by prefixing them with If I can't get rid of the duplication, I rather have it in the But thanks for the suggestion!
I agree and can see that for monster legacy projects doing this is madness, but for a small greenfield project trying to use modern Python features, I think for |
True. For the cases where you use a type-checker, you code will never run in runtime, so that issue should never happen too. This mostly applies to all static checks performed by a type-checker, unless you add runtime In any case, if this feature is controversial, or could be useful only to a subset of users, maybe it can be added behind an option to let users decide what's best for them. |
Ha, this is not an easy topic. So many things to consider here. Let me try to start again. First I'll describe what Griffe currently does. Then I'll share a few observations. Griffe has a concept of "exported member". A module member is considered:
A class member is considered implicitly exported, the same way as module members are considered implicitly exported. It is never considered explicitly exported because we don't/can't use This "exported" marker is then used in a few places, 3 of of them being of interest to us here. 1. when expanding wildcard imports
I see a first issue here: members starting with 2. when checking for breaking changes in the APIBetween the same old and same new module, if a member disappeared, and it was either a submodule, or it was exported (explicitly or implicitly), we consider its disappearance a breaking change. Again, it's wrong here to consider that a private object's disappearance makes a breaking change. I didn't spot this issue before because the code that checks breaking changes has an earlier condition on the object names: it skips private objects anyway. 3. when telling if an object is publicGriffe objects have an
Griffe gained this method later on, to be used by mkdocstrings-python. A few observations. I think Griffe's internals/API is a bit confusing. "Exported" is usually used to say "public" (we export an object to make it public), but in Griffe it's rather used to say "the object is exposed in regard to the wildcard import runtime mechanism". So maybe I should rename "exported" to "wildcard exposed". Then we should fix the issue mentioned above about private members being considered implicitly exposed. And since wildcard imports are not configurable, we would drop the distinction between explicitly and implicitly exposed. An object is either exposed, or it's not. Finally the code that checks breaking changes should use the "public" API instead of the "exposed" one. Now to go back to the subject at hand: whether to support the two conventions Thanks to the above analysis, and the changes we will make, I think it is safe to support them 👍
My comment above about convention vs runtime was just me being confused, or the official docs being unclear. "Imported by wildcard" and "considered public" are not the same thing, and we most probably don't ask type checkers to use the "public" logic when expanding wildcard imports. Just a final note: Griffe is not a type-checker, so type-checker recommendations do not always apply to us. In this case it seems like we can apply them safely 👍 |
Thanks for the very in-depth analysis! Indeed it seems like there is a lot to consider, but I'm glad it seems like these conventions fit into the current Griffe model. In particular I'm not sure I understand the difference between public, exported and exposed are this context. Also now I'm curious about what this BCD you are talking about is! Is is a tool that one can use to check if some changes break the public interface? Or is it just some internal implementation detail from Griffe that's not exposed to "regular users"?
I understand technically this is true, but if your documentation doesn't match what the type checker thinks, this documentation won't be all that useful. 😇 |
Of course 😄 (Just to give one example, type-checkers will apparently only read stubs, while we have to read both regular modules and their stubs, then merge their contents 🙂)
Glad you asked 😎 It's definitely exposed to users, see our docs here: https://mkdocstrings.github.io/griffe/checking/ |
This is amazing, thanks a bunch! |
Just for completeness, here is another more or less official mention to how public and private symbols should work for libraries: |
Problem
The
__init__.py
is sometimes used to re-export functions and classes defined within sub-modules, generally for visibility within the library and also with the auto-completion. It would be nice if the documentation showed this as well.Solution
Ideally, I would like an option that shows explicit imports/re-exports located within
__init__.py
. Based on other linting conventions, the way this is done is to be explicit with anas
even if it is tautological, though may not always be.As for the implementation, I like the way Rust documentation shows re-exports in a section dedicated for re-exports. This makes it clear that the class or function is defined elsewhere but has been made available within the current namespace. Here's an example from the popular Clap library:
Alternatives
I tried looking at the options available, but could not identify anything that would provide the functionality I need.
Another alternative would be to have the
gen-files
script handle this, but the implementation does not appear to be easy.Extra
The ability to document re-exports I think could be implemented with the following options if there might be a need for that, though I struggle to see the use-case for any of them other than
explicit-init
.explicit-init
: This would only document explicit re-exports (i.e., ones that use theas
notation) within__init__.py
files.explicit-all
: This would document explicit re-exports, but within any Python file.implicit-init
: This would document all imports (whether they use the explicitas
notation or not), but only within__init__.py
implicit-all
: As withimplicit-init
, but for all files.The text was updated successfully, but these errors were encountered: