Generalized/WMW odds ratio

[arcaldwell49]

Describe the solution you'd like
I recently have been thinking about using odds ratios to describe the effect size for simple ordinal comparisons (i.e., situations where a Wilcoxon-Mann-Whitney test would typically be used). I have come across a few papers (This by O'Brien and Castelloe being the most convincing that this is worthwhile), but haven't found much support, package wise, for this effect size in R. I think a simple "wmw_odds" function would complement the other non-parametric effect sizes well.

How could we do it?
I think the function could operate similar to wilcox.test with the output instead being the WMW or "generalized" odds ratio (maybe an option of as_hest so it appears like the output of wilcox test rather than a tibble).

Overall, I have the idea and some pseudo code written to implement this. I can fork the project and work on this, but I wanted to gauge interest from the authors/maintainers before I spend a substantial amount of time working on this.

So, would there be any interest in a wmw_odds function?

Cheers,

Aaron

[mattansb]

@arcaldwell49 This looks great! I see they also provide a method for estimating CI - that's good (:

If you would like to make a PR we can get to work on this.

[arcaldwell49]

Awesome! I'll put together a PR in the coming weeks. The only problem I forsee is with the paired/one-sample data, but it should be doable (or I send up the bat signal here and elsewhere for help)

[mattansb]

Note sure how this is related - https://users.stat.ufl.edu/~aa/articles/agresti_1981.pdf Has:

1. Generalized OR
2. Generalized Cliffs delta

One of these is Freeman's theta, maybe?

[arcaldwell49]

Yes, Freeman's theta is mentioned on the top of page 527. The Agresti paper is great, but his method does not account for ties. The appendix is nice because the method described could be used for calculating a confidence interval, but this could not be applied to the single/paired samples comparison. I think I am going to go with BCa bootstrap approach for the CI method, but I need to run a few simulations to ensure the coverage is at least adequate.

[mattansb]

Generally, if there is a defined analytical method for CIs, it should be preferred (either via ncp t/F/chi2 or some normal approximation) - we only resort to bootstrap CIs when such methods are not available.

[arcaldwell49]

Agreed, it has just been difficult to find a solution for that would cover both the paired and two-sample cases. There may be a normal approximation that works but I would need to verify that it has good coverage. There is a way to convert Odds --> Harrell's c --> Somer Dxy then use the Fisher transform, but I already know that the coverage there is not great. I may be able to find a solution using calculation of the variance for an AUC. I will let you know what I find (hopefully soon).

[arcaldwell49]

Okay, I think I have a normal approximation solution for both paired and two-sample cases. However, I see the there is a "ci_method" method argument in some easystats/effectsize package's functions. Would it be okay for me to allow the option to set the ci_method to "normal" or "bca" (BCa bootstrap)? Personally, I'd like to have the BCa option built into the function.

Just trying to conform to the rules and standards ya'll have developed for the package 😄

[mattansb]

Hey, just wrote this up: https://github.com/easystats/effectsize/wiki/Class-%60effectsize_table%60

Let me know if this does/n't answer your Qs (:

[arcaldwell49]

Perfect, I have the function conforming to those standards (I think).

I have the normal approximation working (so method = "normal"), but would it be okay to have a option for the user to set the ci_method themselves (i.e., multiple options for CIs)?

[mattansb]

What other methods have you got in mind?

I think we should start with what you've got as work from there.

[arcaldwell49]

Yeah, my only concern is the method provided by Agresti doesn't provide good coverage outside of small to moderate effect sizes.

A set of simulations I've run based on the function are below (x-axis is total sample size and facets are effect sizes)

Let me look into the BCa intervals. If they don't improve coverage then I won't pursue them.

[arcaldwell49]

Nevermind, the bootstrapping techniques don't improve the situation much. Sticking with the normal approaches. I should have a pull request ready in the next week.

[mattansb]

Closed in favor of #500