Polish for profiling

[palday]

I'm currently reviewing the new profiling code over in #639 and I think it's looking very nice. I think there are some interface features that we should discuss a bit after merging that but before releasing/declaring the API there stable. I've decided to put this into a discussion so that each of my questions/ideas/suggestions can have a separate answer thread.

[palday]

What should the return type of confint(::MixedModelsProfile) be?

Potential options:

• DictTable
• StatsBase.CoefTable
• whatever confint(::MixedModelBootstrap) returns (don't remember off the top of my head)
• Simply a guarantee to be Tables.jl-compliant (which all of the above are), with the exact return type an implementation detail that can change in minor but not patch releases?

It looks like GLM.jl returns a matrix: https://github.com/JuliaStats/GLM.jl/blob/9f5276efdb52d6262a531785e8d9139a99e8b9fd/src/glmfit.jl#L257-L259

[palday]

Should the rownames for confint(::MixedModelsProfile) be changed to match coefficient names for the betas and maybe something like sigma_group_betaname for variance components?

[palday]

What happens when a rank deficient model is profiled?

We should add tests for both

• singular fits
• rank deficient fixed effects

[palday]

When do we expect profile confidence intervals to have better coverage than the parametric bootstrap and vice versa? In my experience, the usual advice is that the bootstrap has the best coverage but I think we've seen a few edge cases in testing where the profile intervals were broader.

[dmbates]

Because the bootstrap confidence interval is evaluated as the shortest interval in the sample with the desired coverage I suspect that it will be too short. That is, if we start from a given model and set of 'true' parameter values then simulate data, fit the model to each simulated data set and evaluate the bootstrap ci's we will find that the coverage is less than the nominal value. It's a consequence of using the length as the property determining the interval.

[palday]

Should we reduce the size of the initial step in computing the profiles? p? The default initial step probably takes us quite far from the optimum, assuming that the optimum in the profiling steps is close-ish to the MLE.

[palday]

Should the profiling functions take a progress argument and then when progress=true, should some @info updates and pass progress to the fit! calls?

[palday]

Do we need a nicer show(::MixedModelProfile) method?

[palday]

What about profiles and Wald CIs for GLMM?