[WIP] Native TOML config #3309

ziima · 2024-07-14T18:49:15Z

This is a prototype solution for #999, see discussion there.

This branch will be rebased a lot.

gaborbernat · 2024-07-14T19:56:02Z

src/tox/config/source/toml.py

+        return f"{type(self).__name__}(path={self.path})"
+
+    def transform_section(self, section: Section) -> Section:  # noqa: PLR6301
+        return IniSection(section.prefix, section.name)


We should not use the ini system at all.

I duplicated IniSection to TomlSection for the time being. I plan to do more clean up later on.

gaborbernat · 2024-07-14T19:56:49Z

src/tox/config/source/toml.py

+            if in_section.prefix is not None:  # no prefix specified, so this could imply our own prefix
+                yield IniSection(in_section.prefix, a_base)
+
+    def sections(self) -> Iterator[IniSection]:


It makes no sense why have anything to do with the ini system here.

gaborbernat · 2024-07-14T19:57:50Z

src/tox/config/source/toml.py

+    This is experimental API! Expect things to be broken.
+    """
+
+    CORE_SECTION = CORE


I don't see why we should reuse the ini systems prefixes for both the core and testing environment.

gaborbernat · 2024-07-14T19:59:08Z

tests/config/source/test_discover.py

@@ -10,7 +10,7 @@

 def out_no_src(path: Path) -> str:
    return (
-        f"ROOT: No tox.ini or setup.cfg or pyproject.toml found, assuming empty tox.ini at {path}\n"
+        f"ROOT: No tox.ini or tox.toml or setup.cfg or pyproject.toml found, assuming empty tox.ini at {path}\n"


I'm not sure I agree with the need for a dedicated TOML file. I am currently -1 on adding that.

I am in favour of defaulting to pyproject.toml but still allowing user to define an alternative config file to load. Some might find a need for it (like CI/CD, or custom user modified one when they cannot merge their changes in the default one).

I would say that even with alternative filename, I would still expect it to follow the same sections as the pyproject.toml, just to keep code simple.

I went through the docs for several non-build tools I know (excluded those which do not support pyproject.toml at all) to see if the support custom TOML file. Here are results:

Tool Custom TOML file Custom TOML root

mypy none

ruff ruff.toml none

bump-my-version .bumpversion.toml tool.bumpversion or bumpversion

isort none

black none

codespell none

coverage none

towncrier towncrier.toml tool.towncrier

I'd say custom TOML file is not that rare. Custom root in TOML file seems to vary.

gaborbernat · 2024-07-14T19:59:40Z

tests/config/source/test_source_toml.py

+    )
+    assert custom_section["a"] == "b"
+
+


I'm not a fan of this long comments.

ziima · 2024-07-15T19:55:52Z

I've ditched TomlLoader and updated MemoryLoader instead. They were mostly the same.

gaborbernat · 2024-07-16T08:37:29Z

src/tox/config/source/toml.py

+
+
+class TomlSource(Source):
+    """Configuration sourced from a toml file (such as tox.toml).


I do not want a dedicated toml, but should be inside pyproject.toml.

I keep that in mind, I just didn't get to that yet.

Personally I'd like to have tox.toml anyway. Tox configurations may get quite complex and I tend to keep them in a separate file, just one in format with a (better) specification. Currently, tox configuration can be written in setup.cfg, but I don't remember a single project where I would see that. From a contributor point of view, when I see tox.ini I know that the project uses tox and I don't really have to look up how to run tests.

I'd prefer to have an option of separate TOML config file as well. If you don't like a dedicated file such as tox.toml, what about allowing to specify configuration file in env variable, such as TOX_CONFIG?

I don't like the environment idea 😃

ziima · 2024-07-27T20:03:24Z

I changed testenv in TOML to tool.tox.env as suggested in #999 (comment)

Currently supported pyproject.toml looks like

[tool.tox]
env_list = ['py311', 'py312', 'type']

# The default testenv is `testenv`.
[tool.tox.env.testenv]
deps = 'pytest'
commands = ['pytest tests']

[tool.tox.env.type]
deps = 'mypy'
commands = ['mypy src']

I also added support for native pyproject.toml. It broke ability to load legacy pyproject.toml using --conf though. To be fixed later.

I split BaseSection from Section to handle different structure of data within TOML. I'm not quite sure what to do with that at the moment.

I keep tox.toml support at least for tests during development. It's easier that way.

gaborbernat · 2024-07-27T21:11:16Z

I think deps needs to be a list of strings. Also, I would like to rename the base environment name to something like testenv -> base.

ziima · 2024-08-11T10:01:56Z

Ad options: deps just needs a tweak to work, but there is an underlying issue regarding factors. I'm not yet sure how to handle factors in TOML. For deps, I could just do

deps = ["quality: ruff"]

but that is somewhat clumsy and it really doesn't work with non-string options, such as ignore_errors.

I'm starting to think the best approach would be to provide a subkey to explicitly define factors in TOML, such as

[[tox.deps]]
name = "ruff"
when = ["quality"]

[[tox.deps]]
name = "pip"
when = ["py312", "py311"]

[[tox.deps]]
name = "setuptools"

# or a shortcut of
[tox]
deps = ["ruff", "pip", "setuptools"]
# when none of the deps has custom factors.

Ad base environment name: I tried to respect the base option, which has testenv as default value. I haven't polish this part, but this is the goal for the moment.

gaborbernat · 2024-08-11T15:28:44Z

IMHO we should do something similar to what hatch does for environment generation and factors. I am ok with having dedicated toml.

gaborbernat · 2024-09-07T05:06:56Z

@ziima do you plan to finalize this?

ziima · 2024-09-07T05:54:38Z

I'm back from holiday, so I should be able to find some time to work on this again.

gaborbernat · 2024-09-07T05:56:55Z

Great to hear so. And thank you for doing all the hard work.

for more information, see https://pre-commit.ci

gaborbernat · 2024-10-01T02:00:30Z

Replaced by #3353

ziima requested a review from gaborbernat as a code owner July 14, 2024 18:49

ziima mentioned this pull request Jul 14, 2024

native pyproject.toml config #999

Closed

gaborbernat requested changes Jul 14, 2024

View reviewed changes

gaborbernat requested changes Jul 16, 2024

View reviewed changes

ssbarnea added the enhancement label Aug 8, 2024

gaborbernat marked this pull request as draft August 13, 2024 16:40

ziima mentioned this pull request Sep 13, 2024

Prepare loaders for TOML #3343

Open

ziima mentioned this pull request Sep 23, 2024

Prepare Loader and MemoryLoader for TOML support #3344

Closed

5 tasks

ziima and others added 6 commits September 27, 2024 12:31

[WIP] Support tox.toml

9ce3ab0

Merge TomlLoader and MemoryLoader

19ff49d

[pre-commit.ci] auto fixes from pre-commit.com hooks

80795f0

for more information, see https://pre-commit.ci

Duplicate IniSection to TomlSection

ba8bb80

Use tox.env.XXX testenvs in TOML, add native pyproject.toml

7365f1e

[pre-commit.ci] auto fixes from pre-commit.com hooks

5473846

for more information, see https://pre-commit.ci

ziima force-pushed the native-toml branch from 1f9d075 to 5473846 Compare September 27, 2024 10:32

pre-commit-ci bot and others added 2 commits September 27, 2024 10:32

[pre-commit.ci] auto fixes from pre-commit.com hooks

0df06a9

for more information, see https://pre-commit.ci

Add conditional settings

49fef5a

gaborbernat closed this Oct 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Native TOML config #3309

[WIP] Native TOML config #3309

ziima commented Jul 14, 2024 •

edited

Loading

gaborbernat Jul 14, 2024

ziima Jul 21, 2024

gaborbernat Jul 14, 2024

gaborbernat Jul 14, 2024

gaborbernat Jul 14, 2024

ssbarnea Aug 8, 2024

ziima Aug 10, 2024

gaborbernat Jul 14, 2024

ziima commented Jul 15, 2024

gaborbernat Jul 16, 2024

ziima Jul 16, 2024

stinovlas Jul 24, 2024

gaborbernat Jul 24, 2024

ziima commented Jul 27, 2024

gaborbernat commented Jul 27, 2024

ziima commented Aug 11, 2024

gaborbernat commented Aug 11, 2024 •

edited

Loading

gaborbernat commented Sep 7, 2024

ziima commented Sep 7, 2024

gaborbernat commented Sep 7, 2024

gaborbernat commented Oct 1, 2024

Tool	Custom TOML file	Custom TOML root
mypy	none
ruff	`ruff.toml`	none
bump-my-version	`.bumpversion.toml`	`tool.bumpversion` or `bumpversion`
isort	none
black	none
codespell	none
coverage	none
towncrier	`towncrier.toml`	`tool.towncrier`



		class TomlSource(Source):
		"""Configuration sourced from a toml file (such as tox.toml).

[WIP] Native TOML config #3309

[WIP] Native TOML config #3309

Conversation

ziima commented Jul 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ziima commented Jul 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ziima commented Jul 27, 2024

gaborbernat commented Jul 27, 2024

ziima commented Aug 11, 2024

gaborbernat commented Aug 11, 2024 • edited Loading

gaborbernat commented Sep 7, 2024

ziima commented Sep 7, 2024

gaborbernat commented Sep 7, 2024

gaborbernat commented Oct 1, 2024

ziima commented Jul 14, 2024 •

edited

Loading

gaborbernat commented Aug 11, 2024 •

edited

Loading