Performance improvements for index generation

[delucchi-cmu]

Change Description

Related to astronomy-commons/lsdb#108

This improves the performance of generating indexes for very large datasets.

• adds optional division hints for setting the new index
• adds option to skip drop_duplicates operation
• sneaks in a quality-of-life improvement when running pytest on epyc: the last test would often fail as a timeout.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (862ea03) 99.74% compared to head (ea6c049) 99.74%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #202   +/-   ##
=======================================
  Coverage   99.74%   99.74%           
=======================================
  Files          24       24           
  Lines        1166     1176   +10     
=======================================
+ Hits         1163     1173   +10     
  Misses          3        3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dougbrn]

Thanks for spending some time on the groupby suggestion, I think this looks good!