Skip to content

Conversation

@gendx
Copy link

@gendx gendx commented Oct 11, 2025

Paralight is a lightweight parallelism library tuned for indexed structures such as slices. Given that the internal representation of hashbrown's hash tables is a slice of buckets (that each optionally contain a value), it's a good fit to integrate with (gendx/paralight#5).

This pull request is here to iterate on the design. As the integration needs access to the raw hash table representation, it's done here in the hashbrown crate (similarly to Rayon's integration).

@clarfonthey
Copy link
Contributor

I think that it's fair to do this given the presence of a Rayon implementation also (although, IMHO, we shouldn't be including these implementations in this crate…) but I also think that regardless of what is done, any primitives needed to make this work should be added to HashTable directly so that people can code their own versions of this.

For example, at one point I was contemplating offering an API that directly provided access to the &[MaybeUninit<T>] and &[Tag] slices in the table, and I think such an API might be helpful here too. Note that the tags and items in the hash table are actually in reverse order to each other, since one uses negative offsets from the central pointer and the other one uses positive offsets.

@Amanieu
Copy link
Member

Amanieu commented Oct 20, 2025

I talked about this in person with @gendx at EuroRust. The main issue with the current implementation is that it returns an iterator of Option<&(mut) T> instead of &(mut) T. I think this should be addressed before paralight supported is added to this crate.

A mid-level API that just exposes buckets like #613 might work for par_iter but would be insufficient for par_iter_mut and into_par_iter. You would need a more complete low-level API as proposed in #545 for that, but it would fundamentally be unstable since it exposes too much of the internal layout of the hash table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants