Release 23.2

[pradyunsg]

That's due in the next month. Filing an issue to track this! :)

[pradyunsg]

@pypa/pip-committers Any takers for the release manager role this time around? :)

[pfmoore]

As I'm already expressing opinions on what should go into the release, I guess I should volunteer 🙂

[pradyunsg]

Sounds fair to me. :)

[pfmoore]

I'm expecting to do the release the weekend of 15/16 July (just under 2 weeks from now). I'm not available the following weekend, and I don't want to leave it until the last couple of days of the month, so that's what works best for me. If anyone has strong arguments for a different release date, speak up, but be aware that it will almost certainly mean a midweek release.

If anyone wants to do a vendoring upgrade, that would be appreciated. Otherwise I'll do one in due course. Anyone involved with or interested in any of the issues on the release 23.2 milestone, please start looking at what needs to be done to get them over the line in time. And if anyone is aware of anything else that they want in 23.2, get it merged ASAP - I'd rather not be dealing with last minute merges.

As usual, I'm following our standard policy of "we release what's on main, with the expectation that main is always in a releasable state". Please don't merge anything that you wouldn't want to see released - if you make me grumpy, I will tell my family who made me hell to live with¹ 🙂

Footnotes

1. Technically, "more hell to live with than normal"... ↩

[pfmoore]

OK, there are no items outstanding on the 23.2 milestone, so I'm starting the release.

[pfmoore]

... and the release is done! That was nice and easy - thanks everyone for your efforts. Let's hope we don't get any major issues with it!

[pfmoore]

Important Note

Python 3.12 RC1 is being released on the 31st July. In order to ensure we don't miss that deadline, I will be taking a very hard stance on bugfixes for this release - anything we want to handle in a bugfix (i.e., before 23.3) will need to be reported, and a fix prepared, in the next week, to give me time to prepare a bugfix release. I will strongly prefer reversion of any offending code over attempts to fix the issue, for anything but trivial fixes.

Worst case scenario will be that Python 3.12 ships with pip 23.1.2. That's not the end of the world, but I'd rather avoid it if at all possible.

[pfmoore]

To make any potential bugfix easier, please avoid merging anything to main for a week or so, unless absolutely necessary.

[pradyunsg]

Ooops! I accidentally merged #11382 out of habit. Lemme know if you'd prefer I revert that @pfmoore. 😅

[pfmoore]

No, it's fine. If we have a bugfix release (here's hoping we don't!) it can go in, as it's a purely internal change. My point was mainly about anything that has user-facing behaviour. In reality, though, as long as nothing prevents me from just releasing main as a bugfix release, I don't mind that much - I just don't have much free time next weekend, so I don't want to have to cherrypick.

[uranusjr]

Looks like we do have an issue (with several reports) on the legacy resolver failing.

[pfmoore]

I'd be inclined to treat it as non-critical, as people shouldn't be using the legacy resolver. Can we check with the people affected why "don't use the legacy resolver" isn't a suitable workaround?

Otherwise, my position remains the same - if someone can contribute a fix this week, I'll release main as 23.2.1. Whatever we have at the end of this week will go into CPython 3.12, and I don't want to issue any further bugfixes once that happens.

Of course, what I wish for might not happen 🙂 But I'd also hate to ship CPython 3.12 with pip 23.1.2, which is what we'll have if we can't get something stable enough to go into the RC. (I guess I might be able to get an exemption to put a bugfix in after RC1, but I'm not particularly willing to try, as I don't think we should expect pip to be treated specially).

[notatallshaw]

I would make the casual observation that the comments so far imply that most people who have legacy resolver turned on don't seem to know why, as they have been quick to turn it off and everything works fine for them.

I would assume users added it as a workaround when the newer resolver was originally turned on by default and never thought about it again.

I do wonder if there's a resolver mode using the new resolver that could be added to cover any remaining valid use cases for the legacy resolver. I'll think on it and make an issue.

[pfmoore]

I do wonder if there's a resolver mode using the new resolver that could be added to cover any remaining valid use cases for the legacy resolver.

Do we have any known valid use cases for the old resolver still? I'm not aware of any. I don't want to do anything to preserve the old behaviour or code unless there's a compelling use case.

[notatallshaw]

Do we have any known valid use cases for the old resolver still? I'm not aware of any. I don't want to do anything to preserve the old behaviour or code unless there's a compelling use case.

The answer to that is going to strongly depend on what you mean by "valid", anyway I wrote up my idea and the context to justify it here: #12160

[pradyunsg]

Do we have any known valid use cases for the old resolver still?

See https://github.com/pypa/pip/milestone/47.

[uranusjr]

Maybe this can be a convenient oppertunity to gauge this. We can start by suggesting people reporting the issue to drop the legacy resolver, and anyone says they can’t we can confidently know the use case.

[pfmoore]

Maybe this can be a convenient oppertunity to gauge this.

Yes, that was my point. But we are on a tight deadline so we can’t (for example) leave it a couple of weeks and then decide we need a bugfix.

[pfmoore]

Looks like we do have an issue (with several reports) on the legacy resolver failing.

Focusing purely on the release management issues here, is anyone looking at creating a targeted fix that does nothing but address the "Error while checking for conflicts" message? I'm not aware of it, if so - everyone seems to be discussing broader questions around the old resolver.

There are only 3 options that really make sense, and I need to choose one of them:

1. Apply a minimal fix that addresses this issue. This isn't possible unless someone provides such a fix.
2. Revert the code that caused the error. This isn't possible unless someone identifies what that code is.
3. Stick with what we have for the 23.2 release, and look at longer-term fixes for 23.3.

There's technically also "extend the deadline for someone coming up with (1) or (2), and ship Python 3.12 with pip 23.1.2". But I don't particularly like that unless someone's already working on (1) or (2) and needs more time to complete it.

I don't personally think it's reasonable to go to the Python 3.12 RM and ask to be allowed to ship pip 23.2 if we intend to release a 23.2.1 during the Python RC period. The pressure to "slip the bugfix version into the RC" would be too high. So we have to treat RC1 (2023-07-31) as our hard deadline. And I'd personally want a soft deadline of the weekend before that (i.e., this coming weekend), so that if I do need to do a 23.2.1 I have some flexibility.

Unless something changes, I plan on going with (3). If any of the @pypa/pip-committers has strong objections to that, please either work on (1) or (2), or suggest an alternative option that I've missed.

[pradyunsg]

Is there any reason we couldn't include a pip bugfix in rc2? Or is that something that you want us to avoid doing, since that'll mean intentional changes during the rc period?

[pfmoore]

Or is that something that you want us to avoid doing, since that'll mean intentional changes during the rc period?

This. I haven't asked @Yhg1s if it's a possibility, but personally, I don't think it's a fair thing to ask. Also, I'm on holiday the latter part of August, so it doesn't buy us as much time as you'd expect.

This comment seems to point to the problem being related to the PEP 714 change. That's the thing that passes partly-prepared requirements around, where I gave up on trying to work out why it bugged out on sdists. I'm honestly not that surprised if it also goes weird with the legacy resolver. I very much don't want to back out PEP 714, and have to tell people we've lost the metadata-only download optimisation again. But I don't hold out much hope of finding a quick fix here - it will need either a lot of my time¹, or someone more familiar with the rat's nest that is the requirement code to come up with something 🙁

Footnotes

1. Probably more than I can reasonably commit to. ↩

[alkasm]

Do we have any known valid use cases for the old resolver still? I'm not aware of any. I don't want to do anything to preserve the old behaviour or code unless there's a compelling use case.

#10614

Would be nice for this to be included with https://github.com/pypa/pip/milestone/47

[pfmoore]

I'm planning on releasing 23.2.1 tomorrow, with #12163 included (and anything else that's on main - I won't be cherrypicking commits for the bugfix). I'll then add that to the CPython ensurepip upgrade in readiness for Python 3.12RC1, and at that point I expect 23.2 to be complete.

I haven't seen any other issues/fixes that warrant inclusion in 23.2.1, so anything else should now wait for 23.3.

[pfmoore]

OK. I have now updated ensurepip (for Python main, 3.12 and 3.11) to pip 23.2.1. So I'm now going to declare the 23.2 release complete. Thanks everyone for your help, feel free to get things merged now if you've been holding off.