#1316 Update packaging-projects.rst with a tip on good module naming

[LLyaudet]

---

📚 Documentation preview 📚: https://python-packaging-user-guide--1424.org.readthedocs.build/en/1424/discussions/choosing-a-module-name/

[LLyaudet]

#1316

[henryiii]

IMO, this is a lot of information for a tutorial. Most users should be following the tutorial, which tells them what to name the files. This is more "guide-like" / advanced information. I think a simple 1-2 sentence warning about folder needing underscores to be importable is fine. (99.9% of users don't need to know about a workaround to import something with a dash in it). In fact, the design of stubs packages with -stubs is specifically to make them obviously not importable in normal Python!

[LLyaudet]

Hello @henryiii :)

I agree there is a lot of informations but I placed it at the end of the tutorial so that it does not overwhelm a person that just wants to follow the tutorial.

Moreover, I think it is useful to have "hubs of links with a small dive on a topic" somewhere.
Somehow, I feel the informations I linked together should be linked together somewhere.
Maybe a better place would be a dedicated page like packaging-projects-advanced.rst or packaging-projects-"something".rst.
In that case, there could be a simple sentence at the end of packaging-projects.rst with a link to this more in-depth page.

Best regards,
Laurent Lyaudet

[jeanas]

I completely agree with @henryiii. @LLyaudet Please refer to the Diataxis documentation process which we use:

https://diataxis.fr/tutorials/

Specifically:

"""
Offer only minimum, necessary, explanation

If the learner doesn’t need an explanation in order to complete the tutorial, don’t explain.
"""

Also see: https://devguide.python.org/documentation/style-guide/#audience

Moreover, having module names that are not valid Python identifiers is an extremely niche use case. I don't think it should be mentioned even in guides outside the tutorial. We should just make it clear that project names are not restricted to be Python identifiers, unlike module names.

[LLyaudet]

Hello @jeanas :)

I don't know what kind of pedagogy study supports this:
"""
Offer only minimum, necessary, explanation

If the learner doesn’t need an explanation in order to complete the tutorial, don’t explain.
"""
When I learn, I always ask myself much more questions than what is required to just follow instructions in a tutorial.
And usually I try to "poke the system" by trying things that are slightly distinct from the tutorial.
And when I teach I always give more context information than the strict minimum.
I understand it may not always be welcome, but that's why I put this tip at the end as I said.
And that I suggested it may be in an "advanced tutorial".

Also see: https://devguide.python.org/documentation/style-guide/#audience

Please be specific. The tone of this link is contemptuous on some part.
Please write here what part of this link you think should apply in this situation.

Best regards,
Laurent Lyaudet

[jeanas]

I don't know what kind of pedagogy study supports this:
"""
Offer only minimum, necessary, explanation

If the learner doesn’t need an explanation in order to complete the tutorial, don’t explain.
"""
When I learn, I always ask myself much more questions than what is required to just follow instructions in a tutorial.
And usually I try to "poke the system" by trying things that are slightly distinct from the tutorial.
And when I teach I always give more context information than the strict minimum.
I understand it may not always be welcome, but that's why I put this tip at the end as I said.
And that I suggested it may be in an "advanced tutorial".

The quote is not implying that documentation should only offer minimum explanation, only that the tutorial should remain minimal, as an entry point for absolute beginners, and all more advanced explanations should live in guides or discussions, with the TOC serving as your “hub of links”. This is merely a way to structure documentation, there are other ways but Diataxis happens to have been chosen for packaging.python.org.

I think that (a) the info you are proposing to add here should not live in the tutorial but in an explanation, following Diataxis, and (b) part of it (specifically, anything related to modules with names that aren't valid Python identifiers) should be stripped.

Also see: https://devguide.python.org/documentation/style-guide/#audience

Please be specific. The tone of this link is contemptuous on some part.
Please write here what part of this link you think should apply in this situation.

Second and third paragraphs. The second asks to “save details for the how-to guides and other sources”.

The third paragraph might sound contemptuous as phrased there, and it is not my intent to express contempt against you, but I do think this is a case of trying to improve the documentation in a way that would have helped you avoid some problem but where that problem isn't actually common. What you see is that you would have benefited from this note. What you may not see is that there are dozens and dozens of other notes of equal or greater importance that could be added to the tutorial, but adding them would make the tutorial difficult to use.

[jeanas]

xref: #1425

[LLyaudet]

Hello @jeanas ,

The quote is not implying that documentation should only offer minimum explanation, only that the tutorial should remain minimal, as an entry point for absolute beginners, and all more advanced explanations should live in guides or discussions, with the TOC serving as your “hub of links”. This is merely a way to structure documentation, there are other ways but Diataxis happens to have been chosen for packaging.python.org.

I think that (a) the info you are proposing to add here should not live in the tutorial but in an explanation, following Diataxis, and (b) part of it (specifically, anything related to modules with names that aren't valid Python identifiers) should be stripped.

Ok, as I commented I was open to have my redaction submitted in this PR moved to a discussion/advanced tutorial, so point (a) is ok with me, if there is at least a small warning with link like "See here for an advanced discussion on package names" in source/tutorials/packaging-projects.rst. But I have see that you somehow decreased the need for that with #1425 and #1426.

But I do not see any valid reason for your point (b).
What is wrong with explaining the possibilities of Python importlib in a more in-depth discussion ?

Let's cite the third paragraph so that navigating is not needed:

Be careful accepting requests for documentation changes from the rare but vocal category of reader who is looking for vindication for one of their programming errors (“I made a mistake, therefore the docs must be wrong …”). Typically, the documentation wasn’t consulted until after the error was made. It is unfortunate, but typically no documentation edit would have saved the user from making false assumptions about the language (“I was surprised by …”).

And your answer :

The third paragraph might sound contemptuous as phrased there, and it is not my intent to express contempt against you, but I do think this is a case of trying to improve the documentation in a way that would have helped you avoid some problem but where that problem isn't actually common. What you see is that you would have benefited from this note. What you may not see is that there are dozens and dozens of other notes of equal or greater importance that could be added to the tutorial, but adding them would make the tutorial difficult to use.

Sorry but I cannot accept this is the right way to see the problem and I still have doubts about your intent.
What I proposed to improve the documentation goes beyond what would have been needed to avoid my problem.
It is an improvement after search of various bits of informations.
Please give examples of other notes of equal or greater importance that could be added to the tutorial, but adding them would make the tutorial difficult to use.
And give examples where adding a F.A.Q. at the end of the tutorial with such notes would make the tutorial difficult to use.
I see no way something at the end of something else can prevent someone from reading the beginning or add complexity to the reading of the beginning.
You see the cathedral in its entirety already ? Or you construct the documentation as a bazaar ?
I think we all know that the current process is bazaar.
It is funny because today this blog post was on HN:
https://olano.dev/2023-11-30-code-is-run-more-than-read/.
The "user" is the person who reads the tutorial with or without mistakes from its part.
Hence listening should be the norm, as improving with the returns of the users.
What is the frequency of improvements that comes from learning from mistakes ?
And what is the frequency of improvements that comes without any mistakes ?
How many projects are less good that they would have been if the dev/biz/whatever just listened with an open mind.
Third paragraph could have been wrote by a spy agency as in :
http://svn.cacert.org/CAcert/CAcert_Inc/Board/oss/oss_sabotage.html
(Thanks HN again for the reference.)
Clearly this paragraph is golden for (11)(a)(7).
And I think that what you say is also in line with (11)(a)(7).
With a lot of bad faith, you can respond that (11)(a)(4) or (12)(d) applies to what I say.
But in the end, I still see an improvement in the content of this PR, even if it needs to be moved to another file.

Best regards,
Laurent Lyaudet

[LLyaudet]

who is looking for vindication

Moreover, there was no vindication when I opened #1316.
Third paragraph makes a free trial of intent to anyone who makes a mistake.

[jeanas]

I'm sorry but it would simply take me more time than I have to formulate a thoughtful response (esp. given the "doubt about my intent"). I'm going to just let people with commit access decide what to do with this PR.

[webknjaz]

Thanks for the attempt to improve this tutorial! We can't accept this change as is, though.

This change is made to a tutorial document, but this document type is expected to include simple instructions exclusively, exactly as is, without any deviations. It probably fits guides or discussions better. Frankly, #1425 is probably what the guide needs instead.
I'm also concerned that this mostly duplicates the explanation of how Python itself works, which isn't directly related to packaging.

Like many other projects over at least half a decade, we're doing our best to follow the Diátaxis framework for writing technical documentation. I witnessed it gaining a rising popularity since at least 2018, since it's formalized and preached by a respected member of the Python community. Some projects/organizations have their own variants of techwriting guidelines that mostly resemble Diátaxis or are direct derivatives of it.
In PyPUG, we also aspire to follow it. Please, refer to https://diataxis.fr which has a lot of information about the different aspects of using it.
Moreover, you can find the author on the Write The Docs Slack instance where there's a corresponding channel: https://www.writethedocs.org/slack/. Daniele is quite responsive there, and this place has a very high concentration of experienced techwriters, exchanging ideas and discussing practical applications of this framework and other things (in separate channels).

I'm afraid, this issue is not a place for debating what Diátaxis got right and wrong. What's important here is that the maintainers and frequent contributors agree that this is the way to go and everybody is expected to respect the convention.

[LLyaudet]

Hello,
I corrected the points given in the review.
Best regards,
Laurent Lyaudet

[webknjaz]

I corrected the points given in the review.

I wouldn't say so. The references are invalid (they must be identifiers Sphinx understands, not filenames). The location of the changes hasn't changed either. Both are hard blockers.

[LLyaudet]

Hello,

I corrected the points given in the review.

I wouldn't say so. The references are invalid (they must be identifiers Sphinx understands, not filenames). The location of the changes hasn't changed either. Both are hard blockers.

Sorry, I corrected the references. It should be ok now.
I switched the content in a new discussion Choosing a Module Name.
And linked between it and Packaging Python Projects.
Let me know if you see anything else to improve :).

Best regards,
Laurent Lyaudet

[LLyaudet]

Hello :),

I tried to follow your suggestions but it is now slightly conflicting.
@henryiii talked about an (advanced) guide.
@webknjaz asked to merge it in another discussion.
@sinoroc thinks a short tutorial would be better.
XD
I'm a little confused in case it was not the goal ;) XD.
Regarding constructive improvements I am able to do, I will try to find the time to test my original mistake with other build tools.
It should be as easy as : checkout my repo django-monkey-patches at one of the first commits, edit the pyproject.toml, build, and inspect the tar files.
I will try to cover all the build tools given in the tutorial.
This way, I can update to make it less focused on Hatch and as generalist as the original tutorial.
If you see other small incremental improvements, tell me.
I hope to do that before next Sunday.
In the meantime, maybe #1426 will have been merged and the correct direction will be clearer.

Best regards,
Laurent Lyaudet

[sinoroc]

@LLyaudet Thanks for working on this.

Clearly we are not sure under what form this topic should appear in the guide and where. I believe @henryiii wanted only to extract it out of the "packaging tutorial", which is now done, so it is good. And I believe @webknjaz would prefer not to have two "discussion" documents on the same topic (#1426). So my suggestion is to make it into some kind of short "how-to guide" document (I suggested "tutorial" in an earlier comment, but I meant "guide"). This way we would have a tutorial, a specification, a discussion, and finally a guide (which is what Diataxis is all about).

If we can agree on this, then we still need to find a nice way to structure and phrase all of this. Maybe @willingc can help with this part.

[sinoroc]

Suggested change

	Make sure to have a valid :ref:`Python identifier <python:identifiers>` for your import package/module name.
	Make sure to choose a valid :ref:`Python identifier <python:identifiers>` for the names of all your :term:`import packages <import package>` and :term:`modules <module>`.

[LLyaudet]

Sorry for the late answer. I validated your commit suggestion of the 4th. The 3rd december I modified like this:

Make sure to have a valid :ref:`Python identifier <python:identifiers>` for your import package/module name.
The (PyPI) :term:`project<Project>`/:term:`distribution package <Distribution Package>` name
and the Python :term:`import package <Import Package>`/module may differ slightly.

So the terms are already referenced.
I found it nice to have all 3 terms referenced in the same sentence when I modified.
But I understand the need to reference one of the 3 terms in the preceding sentence since it is asked here:

packaging.python.org/source/contribute.rst

Line 255 in e8a9716

The first time you mention a tool or practice, link to the part of the

I have a question: Should I reference the glossary on each appearance of a term in the same document?
I know that when a document is long, it is nice when you can hover over any acronym and check again its meaning if it does not stuck the first time.
I think on Wikipedia, it is frequent that most occurrences are cross-links.
As of now, I kept the second reference in the following sentence.
Let me know if I should remove it.
Moreover, if there is a consensus on this question, I think it should be in contribute.rst :)

[LLyaudet]

Hello :),

Thanks for the suggestions @sinoroc :).
It is a little late. I will look at it in detail, tomorrow I hope.
This evening, I did this : https://github.com/LLyaudet/testpypa
There is 8 subfolders: (Flit, Hatchling, PDM, setuptools) times (kebab-case, snake_case)-for import packages.
Each subfolder corresponds to a Project in the glossary.
The structure of each project is the same and roughly that of the tutorial.
There is only 5 commits : the last commit is the result of the build of the 8 projects.
Only one project failed to build : Flit crash when the import package has kebab-case.
For the 7 other projects, a distribution package is successfully created,
but it may be not importable after installation as was the case with Hatchling in my original mistake.
It is too late for me to look in detail at the contents of the distribution packages.
Here also, I hope to look in detail tomorrow.
Feel free to look at it before me :)

Best regards,
Laurent Lyaudet

[LLyaudet]

Hello :),

I finished analyzing the build results in testpypa.

The following applies to my test but may differ in other contexts.
The archive tar.gz suffix is always -1.0.0.tar.gz.
The wheel whl suffix is always -1.0.0-py3-none-any.whl.
The dist-info suffix is always -1.0.0.dist-info.
The egg-info suffix is always -1.0.0.egg-info for setup-tools.
All suffixes are kebab-case.

When the correct import package is found, it is always leaved as is (no normalisation): no lowercase, no underscore instead of dash.
whl, dist-info, and egg-info prefixes are always normalized.
Thus, a mismatch between import package name and dist-info after install is the norm if import package name is not normalized.
tar.gz prefix is normalized by Flit, Hatchling, PDM, but not by setup-tools.

• with kebab-case:
  • Flit: Fail;
  • Hatchling: no error but the wheel is broken with src as importable;
  • PDM: success;
  • setuptools: , success;
• with snake_case: success for Flit, Hatchling, PDM, and setuptools

I tried to be synthetic and group similar results.
Not sure what would be interesting for the documentation,
maybe just two sentences:
"If your import package name is not a valid identifier, Flit will crash at build, Hatchling will create a not working wheel with src as import package, both PDM and setuptools will build correctly.
If your import package name is not a normalized valid identifier or if you have many import packages in your project, there will also be a mismatch between .dist-info file and import package(s) name(s)."

It can replace "Hatch will recognize that the module corresponding to the package is abcd_1234 instead of defaulting to src and building a not working wheel."

I'll try to improve the PR tomorrow :).

Best regards,
Laurent Lyaudet

[LLyaudet]

Hello :),

Sorry for the delay.
I modified the contents with most of the suggestions.
I merged your 2 last suggestions @sinoroc but I switched order and wrote it slightly differently.

I have reopened a conversation where I applied your commit suggestion, since I have a question for best practices regarding term references (only one per document or for each occurrence).
It would be nice if that best practice could be explicit in contribute.rst.

Apart from that, I included the results of my testing of the 4 build backends covered in the tutorial.
I am quite happy that it supports and illustrates further the importance of good naming.
I hope you will see this interest also :).

I build the doc with nox and everything is ok.

If you do not mind, I would like to know if it is ok to move again the document as is from discussions/ to guides/.
Or if it is not in a shape that can be considered a guide by now.
I anticipate that you will probably make more corrections and improvements.
I would feel more comfortable to put the document at the right place,
and then let other authors directly commit on the PR or after the PR is merged.
The document as is conveys the informations I wanted to see in it.
I fear that at the end the document will not look like what I wrote and it is time consuming and painful to just validate others modifications.
Sometimes I see them as improvements and sometimes I just see them as rewording which is not pleasant.
I prefer that each person authors its modifications.

Best regards,
Laurent Lyaudet

[jeanas]

Sorry, this got auto-closed because I wrote "Fixes #1424" in #1426 while I meant "Supersedes #1424". Reopening.

I do think that the info added in #1426 (now here) is sufficient, but I will leave it to committers to choose whether to continue this, or to re-close it and also close the associated issue #1316.

[LLyaudet]

Hello :),
Thanks for reopening it.
I do not think it should be closed, since it provides information that are not covered in #1426.
I'm still opened to suggestions to the correct way of giving this information in the final documentation.
At last iteration, we wanted to put it in a guide.
But I had no feedback on my last comment.
Best regards,
Laurent Lyaudet