Create bootstrapped datasets for regression problems?

[JohannesWiesner]

Hi @rishi-kulkarni, I would like to use your package to create a list of bootstrapped datasets (again referring to HCP data), but I noticed that hierarch.resampling.Bootstrapper.fit() wants to have a value for y to define a treatment and control group. However, the HCP-dataset does not have treatment and control groups (in other words: All my analyses are regression problems). Is it still possible to generate bootstrapped datasets using your functions even if there are no groups?

Reminder: I would like to generate n bootstrapped datasets from the HCP dataset. In this dataset, subjects can belong to the same family or even be twins. I need a function that respects this structure so that resampled datasets are similar in that regard.

[rishi-kulkarni]

HI @JohannesWiesner - I'll take a closer look at what hierarch can do this weekend, but in principle yes. I'd encourage you to take a look at these lecture notes as well: https://faculty.washington.edu/yenchic/17Sp_403/Lec6-bootstrap_reg.pdf

They discuss a couple different approaches to bootstrapping regression problems. When I was putting together the hierarch paper, I found that the approach of bootstrapping residuals + permuting the values of the regression coefficient of interest was best at controlling Type I error rate. The confidence_interval function actually does this out of the box.

If you have multiple covariates at the same level in a regression problem, you can produce the correct permutation test by treating the coefficient you're trying to measure as nested within the others on the same level. See this paper for a deeper discussion of that: https://www.tandfonline.com/doi/abs/10.1080/00949650215733