Add new nvtext minhash_permuted API

[davidwendt]

Description

Introduce new nvtext minhash API that takes a single seed for hashing and 2 parameter vectors to calculate the minhash results from the seed hash:

std::unique_ptr<cudf::column> minhash_permuted(
  cudf::strings_column_view const& input,
  uint32_t seed,
  cudf::device_span<uint32_t const> parameter_a,
  cudf::device_span<uint32_t const> parameter_b,
  cudf::size_type width,
  rmm::cuda_stream_view stream ,
  rmm::device_async_resource_ref mr);

The seed is used to hash the input using rolling set of substrings width characters wide.
The hashes are then combined with the values in parameter_a and parameter_b to calculate a set of 32-bit (or 64-bit) values for each row. Only the minimum value is returned per element of a and b when combined with all the hashes for a row. Each output row is a set of M values where M = parameter_a.size() = parameter_b.size()

This implementation is significantly faster than the current minhash which computes hashes for multiple seeds.

Included in this PR is also the minhash64_permuted() API that is identical but uses 64-bit values for the seed and the parameter values. Also included are new tests and a benchmark as well as the pylibcudf and cudf interfaces.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[Matt711]

Thanks, I left some suggestions.

[GregoryKimball]

@karthikeyann and @kingcrimsontianyu would you please share your review of this work? This is a priority feature for 24.12 for NeMo Curator.

• Do you think the testing is adequate?
• Would you please focus most of your time on min_hash.cu?

[Matt711]

Should we delete the minhash and minhash64 functions or wait until 25.02? For reference, they we're added in 24.12 #17021

[davidwendt]

Should we delete the minhash and minhash64 functions or wait until 25.02? For reference, they we're added in 24.12 #17021

I don't think we should remove the names without at least one release of deprecation.
Is there a way to mark them deprecated in python?

[wence-]

I don't think we should remove the names without at least one release of deprecation.

Agreed

Is there a way to mark them deprecated in python?

Yes: see https://docs.rapids.ai/api/cudf/stable/developer_guide/contributing_guide/#deprecating-and-removing-code

[davidwendt]

I don't think we should remove the names without at least one release of deprecation.

Agreed

Is there a way to mark them deprecated in python?

Yes: see https://docs.rapids.ai/api/cudf/stable/developer_guide/contributing_guide/#deprecating-and-removing-code

So really what we plan to do is change the minhash_permuted to just minhash in 25.02 and get rid of the current minhash which takes different parameters. So I'm not sure what the deprecation warning should say exactly.

[Matt711]

I don't think we should remove the names without at least one release of deprecation.

Agreed

Is there a way to mark them deprecated in python?

Yes: see https://docs.rapids.ai/api/cudf/stable/developer_guide/contributing_guide/#deprecating-and-removing-code

So really what we plan to do is change the minhash_permuted to just minhash in 25.02 and get rid of the current minhash which takes different parameters. So I'm not sure what the deprecation warning should say exactly.

Maybe something like this?

diff --git a/python/pylibcudf/pylibcudf/nvtext/minhash.pyx b/python/pylibcudf/pylibcudf/nvtext/minhash.pyx
index f1e012e60e..641af34eab 100644
--- a/python/pylibcudf/pylibcudf/nvtext/minhash.pyx
+++ b/python/pylibcudf/pylibcudf/nvtext/minhash.pyx
@@ -16,6 +16,7 @@ from pylibcudf.libcudf.types cimport size_type
 from pylibcudf.scalar cimport Scalar
 
 from cython.operator import dereference
+import warnings
 
 
 cpdef Column minhash(Column input, ColumnOrScalar seeds, size_type width=4):
@@ -40,6 +41,11 @@ cpdef Column minhash(Column input, ColumnOrScalar seeds, size_type width=4):
     Column
         List column of minhash values for each string per seed
     """
+    warnings.warn(
+        "Starting in version 25.02, the signature of this function will "
+        "be changed to match pylibcudf.nvtext.minhash_permuted.",
+        DeprecationWarning

And then maybe add a sentence to the docstring of minhash_permuted that the name will be changed in 25.02.