Skip to content

[primer] Refactor the primer to use 'pylint.message.Message' #7000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions pylint/testutils/_primer/primer_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,12 @@
import abc
import argparse
from pathlib import Path
from typing import Dict, List, Union
from typing import Dict, List

from pylint.message import Message
from pylint.testutils._primer import PackageToLint

PackageMessages = Dict[str, List[Dict[str, Union[str, int]]]]
PackageMessages = Dict[str, List[Message]]


class PrimerCommand:
Expand Down
29 changes: 17 additions & 12 deletions pylint/testutils/_primer/primer_run_command.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from itertools import chain

from pylint.lint import Run
from pylint.message import Message
from pylint.reporters import JSONReporter
from pylint.testutils._primer.package_to_lint import PackageToLint
from pylint.testutils._primer.primer_command import PackageMessages, PrimerCommand
Expand All @@ -24,20 +25,18 @@ def run(self) -> None:
packages: PackageMessages = {}

for package, data in self.packages.items():
output = self._lint_package(data)
packages[package] = output
packages[package] = self._lint_package(data)
print(f"Successfully primed {package}.")

astroid_errors = []
other_fatal_msgs = []
for msg in chain.from_iterable(packages.values()):
if msg["type"] == "fatal":
# Remove the crash template location if we're running on GitHub.
# We were falsely getting "new" errors when the timestamp changed.
assert isinstance(msg["message"], str)
if GITHUB_CRASH_TEMPLATE_LOCATION in msg["message"]:
msg["message"] = msg["message"].rsplit(CRASH_TEMPLATE_INTRO)[0]
if msg["symbol"] == "astroid-error":
if msg.category == "fatal":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is category the same as type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if GITHUB_CRASH_TEMPLATE_LOCATION in msg.msg:
# Remove the crash template location if we're running on GitHub.
# We were falsely getting "new" errors when the timestamp changed.
msg.msg = msg.msg.rsplit(CRASH_TEMPLATE_INTRO)[0]
if msg.symbol == "astroid-error":
astroid_errors.append(msg)
else:
other_fatal_msgs.append(msg)
Expand All @@ -48,7 +47,13 @@ def run(self) -> None:
"w",
encoding="utf-8",
) as f:
json.dump(packages, f)
json.dump(
{
p: [JSONReporter.serialize(m) for m in msgs]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this serialisation even necessary? Aren't TypedDict dumpable and loadable by default?

Copy link
Member Author

@Pierre-Sassoulas Pierre-Sassoulas Jun 29, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a typed dict at this point, it's an actual pylint.message.Message instance because when we load the json we deserialize.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but we can't we remove both of those? And just store the TypedDict in the json?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could use the TypedDict we introduced in #7077 yes, but Message is used everywhere in the code base.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And there an immutable location in it that I'd like to use later on.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I missed the JSONReporter serialises all of its messages before printing anyway. That makes this necessary.

for p, msgs in packages.items()
},
f,
)

# Fail loudly (and fail CI pipelines) if any fatal errors are found,
# unless they are astroid-errors, in which case just warn.
Expand All @@ -59,7 +64,7 @@ def run(self) -> None:
warnings.warn(f"Fatal errors traced to astroid: {astroid_errors}")
assert not other_fatal_msgs, other_fatal_msgs

def _lint_package(self, data: PackageToLint) -> list[dict[str, str | int]]:
def _lint_package(self, data: PackageToLint) -> list[Message]:
# We want to test all the code we can
enables = ["--enable-all-extensions", "--enable=all"]
# Duplicate code takes too long and is relatively safe
Expand All @@ -69,4 +74,4 @@ def _lint_package(self, data: PackageToLint) -> list[dict[str, str | int]]:
output = StringIO()
reporter = JSONReporter(output)
Run(arguments, reporter=reporter, exit=False)
return json.loads(output.getvalue())
return [JSONReporter.deserialize(m) for m in json.loads(output.getvalue())]