PyCapsule_Import fails when name is in the form 'package.module.capsule'

[lekma]

BPO	32414
Nosy	@ncoghlan, @encukou, @lekma, @skrah, @ericsnowcurrently, @serhiy-storchaka
PRs	bpo-32414: use string after the last dot as the capsule name in PyCapsule_Import #4988 gh-76595: PyCapsule_Import() now imports submodules if needed. #6898

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2017-12-23.09:47:09.382>
labels = ['interpreter-core', 'type-bug', '3.8']
title = "PyCapsule_Import fails when name is in the form 'package.module.capsule'"
updated_at = <Date 2018-05-17.13:34:57.066>
user = 'https://github.com/lekma'

bugs.python.org fields:

activity = <Date 2018-05-17.13:34:57.066>
actor = 'ncoghlan'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2017-12-23.09:47:09.382>
creator = 'lekma'
dependencies = []
files = []
hgrepos = []
issue_num = 32414
keywords = ['patch']
message_count = 15.0
messages = ['308950', '308990', '308994', '315944', '316079', '316123', '316735', '316794', '316795', '316798', '316799', '316805', '316915', '316923', '316927']
nosy_count = 6.0
nosy_names = ['ncoghlan', 'petr.viktorin', 'lekma', 'skrah', 'eric.snow', 'serhiy.storchaka']
pr_nums = ['4988', '6898']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue32414'
versions = ['Python 3.8']

Linked PRs

• gh-76595: Add note on PyCapsule_Import behavior #134022

[lekma]

When capsule name is in the form 'package.module.capsule', PyCapsule_Import fails with an AttributeError (module is never an attribute of package).

[serhiy-storchaka]

PR 4988 will break the case 'module.attr.capsule' if 'module.attr' is not a module.

[lekma]

PR 4988 will break the case 'module.attr.capsule' if 'module.attr' is
not a module.
you are right, but is that a case you would expect in an import
statement?

[lekma]

thinking out loud here: maybe both behavior can be exposed, i.e. a PyCapsule_Import that would expect a valid import statement and a Py_GetCapsule that would behave more like getattr?

[encukou]

An option would be to use a colon to separate the module(s) from the attribute(s). The "inspect" module CLI does this, for example:
https://docs.python.org/3/library/inspect.html#command-line-interface

$ python3 -m inspect urllib.parse:SplitResult.geturl
    def geturl(self):
        return urlunsplit(self)

A colon can't appear in an identifier, so the old way can be used if there's no colon in the argument, making this backwards compatible.

[ncoghlan]

"package.module:attribute" is also the syntax used in packaging tools to unambiguously separate the name of the module to be imported from the attribute chain to be looked up within that module: https://packaging.python.org/specifications/entry-points/

So +1 from me for extending it to this use case as well.

[ericsnowcurrently]

+1 on using ":" as the separator.

[serhiy-storchaka]

Using ":" makes the syntax irregular. "package.module:attribute", but "module.attribute". And "package.module.attribute" works too if "package.module" already is imported.

It is possible to support importing submodules without using ":".

[skrah]

I'm sorry if this interrupts the discussion, but isn't this

https://github.com/plures/xnd/blob/06f6dd82dda9c7bca5df30b121b5cd75c6337609/python/xnd/pyxnd.h#L103

of the form package.module.capsule?

This works, and I just want to make sure it continues to do so.

[skrah]

Ah, Serhiy stated the reason (module already imported) previously. Sorry for the noise.

[serhiy-storchaka]

In your case importing xnd automatically imports xnd._xnd and sets it as an attribute of xnd. This case works now. PR 6898 adds support for cases when a submodule is not imported automatically by importing a package.

[encukou]

Is "package.module:namespace.attribute" worth supporting? I guess it isn't, at least for now. So Serhyi's patch will work.

[ncoghlan]

The behaviour I'd expect to see:

"module:qual.name" -> "imports module, accesses module.qual.name"
"module.qual.name" -> "no implicit import, accesses module.qual.name"
"package.submodule:qual.name" -> "imports package.submodule, accesses package.submodule.qual.name"
"package.submodule.qual.name" -> "no implicit import, accesses package.submodule.qual.name"

So if you have ":" in the capsule path, you're requesting that the interpreter execute an import with the path up to that point before attempting to resolve the full name reference.

By contrast, if you omit the ":", you're telling the interpreter that you'll take care of ensuring that any required imports have taken place before attempting to resolve the capsule reference.

[serhiy-storchaka]

This change would break virtually all existing usages of PyCapsule_Import(). And it would look very surprising taking into account the name of this function.

Currently "package.submodule.qual.name" imports package and accesses package.submodule.qual.name. You are out of luck if package.submodule was not imported before or if importing package doesn't import package.submodule.

[ncoghlan]

Ah, sorry, I misinterpreted Petr's comment, and then didn't look at the details of your PR before commenting.

Having fixed that mistake, I agree that making the naive approach "just work" is a good option.