Consider Dropping Python 3.9 Support and Adopting SPEC0

[rbeucher]

As part of aligning with emerging standards and simplifying our support for newer Python versions, I'd like us to consider dropping support for Python 3.9 and moving towards adopting the SPEC0 standard. SPEC0, as outlined by the Scientific Python ecosystem, sets a foundation for compatibility with modern Python versions while ensuring robustness and performance improvements. You can find more details about SPEC0 here.

Any thoughts @charles-turner-1 @marc-white ?

[marc-white]

I think this is a reasonable standard to aim for, with a couple of caveats:

• Do we know if there is a tool available for checking and alerting us to upcoming end-of-support dates, via, e.g., GitHub CI/CD?
• A particular word of warning with numpy - there were some major changes between v1 and v2 that it may be an idea to get well ahead of.

[rbeucher]

Not sure about tools.
I agree about Numpy v2. We should get support for it.

[charles-turner-1]

Yeah, I'm fully on board with this. Dropping 3.9 support will let us take advantage of structural pattern matching & make some typing improvements (eg. def foo(x : Optional[int]): ... -> def foo(x : int | None): ...).

Structural pattern matching I think will have a lot of really solid use cases here, although I'm yet to give it enough thought to see how we would utilise it - it just makes sense with the intention of the package.

I'm not sure if we need to worry as much about numpy v2 as it might look like on the surface - this package doesn't really do any numeric heavy lifting itself & so any changes/incompatibilities should hopefully be handled by the dependency versions our requirements pin to... hopefully.

[charles-turner-1]

Re. dates, I think that the release cycle for Python versions is pretty set in stone now - happens every October I think. I've had a look and I can't find any API endpoints that would give us some sort of status code, but I think we could create some sort of date tracker ourselves easily enough.

Not sure whether there are other tooling solutions out there. I tried installing a deprecated version of numpy (1.13) in a test environment - conda couldn't find it & pip failed, so perhaps the tooling might handle this for us to an extent.

[rbeucher]

I have moved the conda/access-med-0.10 environment to python>=3.10 and everything should be good.

[charles-turner-1]

I have moved the conda/access-med-0.10 environment to python>=3.10 and everything should be good.

I'll go through this package and drop anything 3.9 related when I get the chance.

[rbeucher]

I just realised today that SPEC0 has now dropped 3.10 too...
Can we dropped 3.10? What do you think @charles-turner-1 ?

[charles-turner-1]

Yeah, I did notice that. I think the conda/analysis3 environment uses python 3.10.14 by default so I think we should hold off for now, but I think we should think about deprecating 3.10 for sure.

[rbeucher]

The hh5 transition plan is to hand over analysis3 to ACCESS-NRI. I had a look and cdms2 is preventing the upgrade to python 3.11.

[charles-turner-1]

Do we need to retain cdms2 for anything? I saw your comment yesterday afternoon but haven't dug any further - looks like it was deprecated over a year ago so it might be time to drop support?

[rbeucher]

I'll start an issue on the COECMS GitHub to ask.

[rbeucher]

See here coecms/cms-conda-singularity#88 (comment)