MRG: core: add scaled selection to manifest; add helper functions for collection and sig/sketch usage

[bluegenes]

This PR adds:

New functions:

• Collection::sig_from_record

when we iterate through the sketches, we have both idx and record available. I thought it would make sense to just use record directly, rather than re-getting record from index.

• Signature::minhash

if there is one minhash sketch available within the sig that matches selection params, return it

• Signature::get_sketch

if there is one sketch (of any type) available within the sig that matches selection params, return it. Note that since this returns the sketch enum, it still requires checking MinHash type afterwards. @luizirber is there a way to return any of the sketches directly from the same function (like minhash function, above, but more flexible?).

• Manifest::From<&PathBuf>

build a manifest directly from a pathlist file. Added and tested, but lmk if you think we should just build a list of paths separately. I wanted this for branchwater, but am not actually using it since neither the paths or PathBuf loading code allow missing/failed paths.

New selection functionality

• added scaled and num selection to manifest. For scaled, if sketch is compatible (equal scaled or can be downsampled), keep it during manifest selection. Otherwise, discard.

Tests added for each new function/added code.

---

Context and prior comments

There are a few things I found difficult to use in sourmash-bio/sourmash_plugin_branchwater#197. This PR is to propose some changes that might simplify things.

In the new core, when we want minhash from a signature (SigStore), we need to go through a couple checks because it's in a sketches vec within in a Sketch enum, However, when we're loading from a collection, like with collection.sig_for_dataset, we know we only have a single sketch (AND we know it's a MinHash type, because we ran Select on it with scaled). Indeed, probably this is the case often, because sig select is doing the work?

from luiz:
ok, let's see:

• the assert_eq! is there because after Select there should be only one matching sketch inside the sig. The selection here is initialized from a Record (a line) in the Manifest, which might have one sig file with different params (common one: k=21,31,51, leading to three lines in the manifest), but since sig_for_dataset takes a dataset_id (which refers to one Record/line), it should always return one sketch in one sig.
• we can have a .minhash() method in Signature as a shortcut, but will have similar logic to load_single_sketch_from_sig. It can be much shorter:

    pub fn minhash(&self) -> Option<&KmerMinHash> {
        for sk in &self.signatures {
            // TODO:  also account for Sketch::LargeMinHash?
            if let Sketch::MinHash(mh) = sk {
                // TODO: might want to iterate over all and guarantee only
                // one minhash was around
                return Some(mh);
            }
        }
        None
    }

Other notes from Luiz to address or file in an issue:

• .sketches() is absolutely terrible and we should find a replacement. It clones all the data in the Signature, which can be a LOT of useless work
• rust and python code should support same ops. Luiz started unifying a bunch of the code in Feature: Mutable/Frozen MH in FFI #2728 but at this point that's very outdated

[codecov]

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (fee6292) 86.60% compared to head (624c7d7) 86.65%.

Files	Patch %	Lines
src/core/src/signature.rs	75.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           latest    #2948      +/-   ##
==========================================
+ Coverage   86.60%   86.65%   +0.04%     
==========================================
  Files         135      135              
  Lines       15264    15284      +20     
  Branches     2592     2592              
==========================================
+ Hits        13220    13244      +24     
+ Misses       1744     1740       -4     
  Partials      300      300              

Flag	Coverage Δ
hypothesis-py	25.80% <ø> (ø)
python	92.82% <ø> (ø)
rust	59.52% <91.30%> (+0.42%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bluegenes]

ok @ctb @luizirber ready for review!

fyi - looks like they python tests are passing but then failing to upload the report to codecov. I'll rerun the tests in a bit.

[ctb]

fyi - looks like they python tests are passing but then failing to upload the report to codecov. I'll rerun the tests in a bit.

that does happen a lot... our codecov stuff is routinely the most brittle part of our CI.

[ctb]

Tests now pass!

[luizirber]

Other than the replacement of dbg! with todo!, this looks great! and so many tests! =]