Refactor Command.run()

[dcermak]

Command.run() currently has a bit of a confusing behavior: if raise_on_error is False and the executable is not found, then a weird CommandT is returned (return code is -1 and stdout+stderr is None). This makes it possible to handle command not found errors separately, but it makes that needlessly verbose. So instead, let's just return None in this special case.

That in turn uncovered, that in most cases when we set raise_on_error=True, we actually want an error if the command is not present but no error if the command fails to execute (e.g. because it returns -1 if you run $cmd --version 🙄). Hence we introduce the flag raise_on_command_not_found, which causes an exception to be raised if the command is not found. This makes it independent of the raise_on_error flag.

Additionally, we add a small optimization: if command starts with /, then we assume it's a full path and we omit the call to which (and just check whether it exists).

These changes require additional tests and a few assertions in places where mypy cannot infer that a certain value is of the correct type.
Also, add type hints in a few (mostly random) places.

[Conan-Kudo]

Umm wow. @schaefi can we have this staged to run image build tests before I approve and merge this?

[schaefi]

Umm wow. @schaefi can we have this staged to run image build tests before I approve and merge this?

yes I will stage this. It's a big change

[schaefi]

Thanks, this is a nice change 👍

I will run the staging builds with it to check if anything breaks. Other than that I only have one question on the use of assert

[schaefi]

I submitted to Staging, let's see what happens :)

[schaefi]

@dcermak The changes works except for distros with py <3.8, there it ends with

[   62s] ImportError: cannot import name 'Literal'

This is easy to fix and as long as it's easy to stay compatible we should do this. Thus can you please add

-from typing import TYPE_CHECKING, Dict, Literal, Union, overload
+import sys
+from typing import TYPE_CHECKING, Dict, Union, overload
+
+if sys.version_info >= (3, 8):
+    from typing import Literal  # pragma: no cover
+else:  # pragma: no cover
+    from typing_extensions import Literal  # pragma: no cover

This will fix it.

Thanks

[dcermak]

@dcermak The changes works except for distros with py <3.8, there it ends with

[   62s] ImportError: cannot import name 'Literal'

This is easy to fix and as long as it's easy to stay compatible we should do this. Thus can you please add

-from typing import TYPE_CHECKING, Dict, Literal, Union, overload
+import sys
+from typing import TYPE_CHECKING, Dict, Union, overload
+
+if sys.version_info >= (3, 8):
+    from typing import Literal  # pragma: no cover
+else:  # pragma: no cover
+    from typing_extensions import Literal  # pragma: no cover

This will fix it.

Thanks

Done. I have also added a test run for both python 3.8 & 3.7, as we'll otherwise get more of these failures in the future.

[schaefi]

@dcermak please drop 3.7 from the matrix. Due to the fb69627 3.7 is out for the tests

[schaefi]

@dcermak I added a commit to skip the 3.7 unit test runs and will create a new Staging build once the green checkmark is back. I'd like to get this one in and wanted to zip a new Staging build from a clean state, hope that's ok

As we cherry-picked a fix to master (for a release to ALP) I also need to fix the compliance check as I think we have to allow-empty there... that's why I add to your work

[Conan-Kudo]

IIRC, we're also dropping Python 3.8 since our new floor will be Python 3.9?

[schaefi]

IIRC, we're also dropping Python 3.8 since our new floor will be Python 3.9?

we said so but there are a few code parts that differentiate on the import for the typing vs. typing_extensions and the boundary here is py3.8. So for the unit test matrix I agree with Dan that we should keep 3.8 in when we have code for it. I don't want to delete the code that differentiates on the typing vs typing_extensions because that also keeps it functional for <3.8 even if that is something we don't care so much. It's very simple to keep that and there is no point in removing imho

[schaefi]

Creating a new Staging build...

[dcermak]

How about we just run mypy for python 3.7? I don't like the idea of having code that we don't test in production, but that will give us at least errors on these kind of import issues.

[schaefi]

How about we just run mypy for python 3.7? I don't like the idea of having code that we don't test in production, but that will give us at least errors on these kind of import issues.

sounds good to me 👍