UP032 has many false positives and incorrect fixes

[dscorbett]

Description

f-string (UP032) has many false positives and incorrect fixes in Ruff 0.9.4. They could be considered multiple unrelated bugs but it might be more useful to reconsider the rule holistically.

With str.format, all the arguments are evaluated before any are formatted, but in an f-string, each argument is formatted immediately after it is evaluated. If formatting an argument has a side effect that affects a later argument, switching to an f-string can change the program’s behavior.

$ cat >up032_0.py <<'# EOF'
from collections import defaultdict
d = defaultdict(list)
print("{[x]} {}".format(d, len(d)))
# EOF

$ python up032_0.py
[] 0

$ ruff --isolated check --select UP032 up032_0.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_0.py
from collections import defaultdict
d = defaultdict(list)
print(f"{d['x']} {len(d)}")

$ python up032_0.py
[] 1

The fix can change the program’s behavior by removing an argument with a side effect.

$ cat >up032_1.py <<'# EOF'
print("{1}".format(x := 1, x))
# EOF

$ python up032_1.py
1

$ ruff --isolated check --select UP032 up032_1.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_1.py
print(f"{x}")

$ python up032_1.py 2>&1 | tail -n 1
NameError: name 'x' is not defined

Another example of a behavior change:

$ cat >up032_2.py <<'# EOF'
print("1".format(1 / 0))
# EOF

$ python up032_2.py 2>&1 | tail -n 1
ZeroDivisionError: division by zero

$ ruff --isolated check --select UP032 up032_2.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_2.py
print("1")

$ python up032_2.py
1

The fix assumes the attribute name in a replacement field is an identifier. When the attribute is not an identifier, the correct fix would involve getattr, in which case switching to an f-string might not be an improvement to readability. When the attribute name happens to be valid syntax, as in the following example, the program might continue running with different behavior; otherwise, you get #7074.

$ cat >up032_3.py <<'# EOF'
from types import SimpleNamespace
x = SimpleNamespace(**{" a": "success", "a": "failure"})
print("{. a}".format(x))
# EOF

$ python up032_3.py
success

$ ruff --isolated check --select UP032 up032_3.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_3.py
from types import SimpleNamespace
x = SimpleNamespace(**{" a": "success", "a": "failure"})
print(f"{x. a}")

$ python up032_3.py
failure

The fix parses an argument index between braces using Rust’s usize::from_str, which allows a leading +, whereas Python’s format strings don’t. A leading + indicates an argument name, not an argument index.

$ cat >up032_4.py <<'# EOF'
print("{+0}".format(0))
# EOF

$ python up032_4.py 2>&1 | tail -n 1
KeyError: '+0'

$ ruff --isolated check --select UP032 up032_4.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_4.py
print(f"{0}")

$ python up032_4.py
0

The fix ought to parenthesize certain arguments that contain colons lest the colons be interpreted as indicating format specifiers.

$ cat >up032_5.py <<'# EOF'
print("{}".format(x := 1))
# EOF

$ python up032_5.py
1

$ ruff --isolated check --select UP032 up032_5.py --fix
Found 1 error (1 fixed, 0 remaining).

$ cat up032_5.py
print(f"{x := 1}")

$ python up032_5.py 2>&1 | tail -n 1
NameError: name 'x' is not defined

A misinterpreted colon can also cause a reverted syntax error.

$ cat >up032_6.py <<'# EOF'
print("<lambda>" in "{}".format(lambda: 1))
# EOF

$ python up032_6.py
True

$ ruff --isolated check --select UP032 up032_6.py --diff

error: Fix introduced a syntax error. Reverting all changes.

This indicates a bug in Ruff. If you could open an issue at:

    https://github.com/astral-sh/ruff/issues/new?title=%5BFix%20error%5D

...quoting the contents of `up032_6.py`, the rule codes UP032, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!

A reverted syntax error can also result from misinterpreted braces.

$ cat >up032_7.py <<'# EOF'
print("{}".format({} | {}))
# EOF

$ python up032_7.py
{}

$ ruff --isolated check --select UP032 up032_7.py --diff 2>&1 | grep quoting
...quoting the contents of `up032_7.py`, the rule codes UP032, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!

A reverted syntax error can also result from misinterpreted quotation marks.

$ cat >up032_8.py <<'# EOF'
x = {"'": "success"}
print("{[']}".format(x))
# EOF

$ python up032_8.py
success

$ ruff --isolated check --select UP032 up032_8.py --diff 2>&1 | grep quoting
...quoting the contents of `up032_8.py`, the rule codes UP032, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!

Given an invalid conversion flag, the fix converts a ValueError to a syntax error. There is no reasonable way to avoid a behavior change for this kind of broken input, so this is a false positive: the rule should ignore it.

$ cat >up032_9.py <<'# EOF'
"{!?}".format(0)
# EOF

$ python up032_9.py 2>&1 | tail -n 1
ValueError: Unknown conversion specifier ?

$ ruff --isolated check --select UP032 up032_9.py --diff 2>&1 | grep quoting
...quoting the contents of `up032_9.py`, the rule codes UP032, along with the `pyproject.toml` settings and executed command, we'd be very appreciative!