MRG: Make kmers_and_hashes iterable

[Adamtaranto]

• Make kmers_and_hashes iterable
• Decide what to do about skip_bad_kmers (force arg in SeqToHashes) - suppress output or error?
• Integrate with count() and consume()
• Init KmerCountTable with option to track kmers
• Add HashMap to store hash:kmer pairs
• Add unhash() function
• Add dump_kmers() function for sorted kmer,count output
• Make dump_kmers() compatible with records dropped by mincut(), maxcut(), drop(), or drop_hash() Close dump_kmers fails after removing records with maxcut #75

Close #21

[Adamtaranto]

@ctb thoughts on bad kmer handling for this function?

skip_bad_kmers is being used to set the force option in SeqToHashes which decides if bad kmers should halt seq processing or just emit a hash of 0.

When skip_bad_kmers is set to True our kmers_and_hashes function will still output a blank kmer with hash 0 rather than actually "skipping" it.

I think we should have the force option in SeqToHashes permanently set to True and then only use skip_bad_kmers to decide if we will output the blank kmer:hash pair as ("",0).

Can you think of any cases where you would want to have force set to False so that only kmers up to the first bad kmer are processed?

[ctb]

I think I agree with you - good point ;)

[Adamtaranto]

Ok, I've updated the iterator so it just prints a warning msg to stderr if it sees a bad kmer, and only return includes empty ("",0) in output if skip_bad_kmers = false.

If this is too noisy then I guess we could turn off warning if skip_bad_kmers = true, though I feel it would be nice to keep as is in case the user does something wacky like try to process protein as DNA.

Can you take a look at the tests? I only know the one method for capturing warning msgs. hammer, nail, etc.

[Adamtaranto]

Previous fix for dependabot ruff action GITHUB_TOKEN prevents jobs from restarting after being interrupted by the ruff action.

reverted.

Will need to disable ruff formatting in any actions spawned by dependabot.

[Adamtaranto]

@ctb ready for review. Do we want to work on making this faster before merging into main?

Low hanging efficiency fruit:

• Revcomp the full seq and step through backwards in sliding window to find canonical kmers instead of revcomp per kmer.
• Make consume multithreaded at the kmer level - both KmersAndHashesIter and SeqToHashes are iterators so this should be fairly simple, unless we are worried about diff threads trying to write to same hash?
• Make multithreaded at the sequence level?
• Consume directly from file to avoid having to load seqs in Python and pass back to rust per WIP: add consume_file #10

[ctb]

excellent!

[Adamtaranto]

Are we good to merge?

[ctb]

yep! go ahead!

[Adamtaranto]

Cool, I'll pull in changes from #82 when tests are finished, then merge.