Pin RAPIDS nightly version in conda installs

[jameslamb]

Description

See #14 (comment) in the parent issue for context.

In short, for any CI scripts doing this (pseudocode):

conda env create -n test --file env.yaml

LOCAL_CHANNEL=$(download-ci-artifacts)

conda install \
    -c "${LOCAL_CHANNEL}" \
    somepackage

That conda install should instead pin to a specific version (e.g. 24.10), like this:

conda install \
    -c "${LOCAL_CHANNEL}" \
    "somepackage=${RAPIDS_VERSION}"

To reduce the risk of issues like those we saw in cugraph near the end of the 24.10 release cycle, where 24.10 builds were silently getting 24.08 or 24.12 nightlies (rapidsai/cugraph#4690).

Benefits of this work

Improves release confidence in RAPIDS libraries.

Reduces the risk of packages from a different RAPIDS release being pulled into CI, which can lead to incorrect packages and docs, silently-unnoticed compatibility issues, and wasted developer time and energy investigating hard-to-understand bugs or CI failures. (related: #22)

Acceptance Criteria

• every RAPIDS repo is pinning its conda install's in CI scripts to a single RAPIDS release (e.g. 24.12)

Approach

Make changes similar to those seen in rapidsai/cugraph#4690 across RAPIDS.

Notes

These updates can be done in any order, since they're self-contained to individual repos.

Updates

Beta Give feedback

1. cucim (make conda installs in CI stricter cucim#791)
2. cudf (make conda installs in CI stricter cudf#17013, make conda installs in CI stricter (part 2) cudf#17042)
3. cugraph (Constrain versions of PyTorch and CI artifacts in CI Runs, upgrade to dgl 2.4 cugraph#4690)
4. cugraph-docs (https://github.com/rapidsai/cugraph-docs/pull/46 (blocked by Changed ci build_docs to just upload the xml so the cugraph-docs can create the docs cugraph#4662)
5. cugraph-gnn (add full CI for wholegraph cugraph-gnn#58, add PR CI for cugraph-pyg and cugraph-dgl cugraph-gnn#59)
6. cugraph-ops (https://github.com/rapidsai/cugraph-ops/pull/699)
7. cuml (make conda installs in CI stricter cuml#6103, fix version in Doxygen docs cuml#6104)
8. cuopt (https://github.com/rapidsai/cuopt/pull/2038)
9. cuspatial (remove unused variables in build_docs script cuspatial#1469)
10. cuvs (make conda installs in CI stricter cuvs#406)
11. cuxfilter (make conda installs in CI stricter cuxfilter#639)
12. dask-cuda (make conda installs in CI stricter dask-cuda#1395)
13. kvikio (make conda installs in CI stricter kvikio#495)
14. private repos (none that need updates)
15. pynvjitlink (make conda installs in CI stricter pynvjitlink#106)
16. raft (make package installations in CI stricter raft#2467)
17. rmm (make conda installs in CI stricter rmm#1696, make conda installs in CI stricter (part 2) rmm#1703)
18. ucx-py (make conda installs in CI stricter ucx-py#1082)
19. ucxx (make conda installs in CI stricter ucxx#298)
20. wholegraph (make conda installations in CI stricter wholegraph#228)

[jameslamb]

I started a small batch of these:

• make conda installs in CI stricter cudf#17013
• make conda installs in CI stricter cuml#6103
• make conda installs in CI stricter rmm#1696

Will see how reviews go on those before continuing with the rest of the projects.

[jameslamb]

Going through these, I found this in cucim:

RAPIDS_VERSION_NUMBER=$(rapids-generate-version)

# ... omitted ...

rapids-mamba-retry install \
  --channel "${CPP_CHANNEL}" \
  --channel "${PYTHON_CHANNEL}" \
  "libcucim=${RAPIDS_VERSION_NUMBER}" \
  "cucim=${RAPIDS_VERSION_NUMBER}"

(build link)

I should have thought of that before! That's even better than using rapids-version-major-minor like I had been. It'll ensure CI looks for something like libcucim=24.12.00a31 or similar... which would prevent cases like "silently fell back to an earlier 24.12.* nightly", like we saw in the beginning of the Python 3.12 migration in the 24.10 release: rapidsai/integration#719 (comment)

[jameslamb]

Thinking more about #106 (comment)... I don't think rapids-generate-version is safe to use like that.

Because it will generate a version like:

cudf=24.12.00a17

Where the 17 means "17 commits from the latest tag to this PR". Running that once at build time and then again, later at test time might result in 2 different values if more commits are merged in the time between those runs (totally possible in a highly-active repo like cudf).

I'm going to use rapids-version to get {major}.{minor}.{patch} and finish this up. It at least is a cheap way to avoid the worst case of this (e.g. using 24.08 packages when you wanted 24.12). Strict channel priority (#84) and combining the installs (#22) in the future will get us stronger guarantees.

[jameslamb]

The only remaining item here was cugrpah-docs, but development on that has been stalled for months:

• https://github.com/rapidsai/cugraph-docs/pull/46#discussion_r1768726948
• Changed ci build_docs to just upload the xml so the cugraph-docs can create the docs cugraph#4662

And isn't planned to be completed in 24.12. There's already that open PR (https://github.com/rapidsai/cugraph-docs/pull/46) tracking this CI-script fiddling for that repo, I don't think it's worth keeping this issue open on this board just for that currently-inactive repo.

I'm closing this, so we can focus on the other remaining 24.12 things.