Exact Bootstrap?

[mdroste]

Hi all - thanks for building out this awesome project! I have a very small beef with the readme (sorry in advance), in particular the exact bootstrap. I worry it's misleading to tell users this feature is useful in practice, for at least two reasons:

1. If we have N=10 unique observations, that's 10^10=10 billion resamples! This method is only computationally feasible when N is in the single digits. Even an incredibly simple estimator (i.e. sample mean) and a huge distributed computing environment can't make this work for N>15 or so.
2. Bootstrap methods all rely on an asymptotic argument that the empirical distribution of the sample 'looks like' the population distribution if the sample is sufficiently large. I don't know any case where resampling from a sample w/ < 10 obs will tell you anything meaningful about the sampling distribution of a statistic. Even in a simple case, like if we observe N=10 draws from a normal distribution (unknown mean and variance) and want to do inference on the mean, a bootstrap on such a small sample won't tell you anything meaningful.

So either the exact bootstrap is computationally feasible but does not answer what you want it to, or it is not computationally feasible.

I see this as a nice thing to have in the library, but I can't see why any practitioner would want to use it.