-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Rausch
authored and
Rausch
committed
Oct 9, 2024
1 parent
b71a398
commit 98e88af
Showing
1 changed file
with
113 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -47,6 +47,119 @@ At the time of writing, `statConfR` is the only available package for an open so | |
|
||
An important limitation of the models implemented in `statConfR` is that the dynamics of the decision process are not taken into account. This is a problem because confidence judgments are related to the dynamics of decision making [@hellmann_confidence_2024; @Pleskac2010; @Rahnev2020]. However, most previously proposed dynamical models of confidence do not include a parameter to represent metacognitive ability. There is one proposal for a dynamical measure of metacognitive efficiency, the v-ratio [@desender_dynamic_2022], which is based on two-stage signal detection theory [@Pleskac2010], but two-stage signal detection theory has been outperformed by other models in a number of visual discrimination tasks [@hellmann_simultaneous_2023; @hellmann_confidence_2024; @shekhar_how_2024]. Thus, the static confidence models included in `statConfR` may still be useful for many researchers. | ||
|
||
# Step-by-step example | ||
|
||
## Installation | ||
|
||
The latest released version of the package is available on CRAN via | ||
```r | ||
install.packages("statConfR") | ||
``` | ||
The easiest way to install the development version is using `devtools` | ||
and install from GitHub: | ||
|
||
devtools::install_github("ManuelRausch/StatConfR") | ||
|
||
## Usage | ||
|
||
### Data structure | ||
|
||
The package includes a demo data set from a masked orientation | ||
discrimination task with confidence judgments (Hellmann et al., 2023, | ||
Exp. 1. | ||
|
||
``` r | ||
library(statConfR) | ||
data("MaskOri") | ||
head(MaskOri) | ||
``` | ||
|
||
## participant stimulus correct rating diffCond trialNo | ||
## 1 1 0 1 0 8.3 1 | ||
## 2 1 90 0 4 133.3 2 | ||
## 3 1 0 1 0 33.3 3 | ||
## 4 1 90 0 0 16.7 4 | ||
## 5 1 0 1 3 133.3 5 | ||
## 6 1 0 1 0 16.7 6 | ||
|
||
Data should be in the form of a data.frame object columns for following | ||
variables: | ||
|
||
- stimulus (factor with 2 levels): The property of the stimulus which | ||
defines which response is correct | ||
- diffCond (factor): The experimental manipulation that is expected to | ||
affect discrimination sensitivity | ||
- correct (0-1): Indicating whether the choice was correct (1) or | ||
incorrect(0). | ||
- rating (factor): A discrete variable encoding the decision confidence | ||
(high: very confident; low: less confident) | ||
- participant (integer): giving the subject ID. | ||
|
||
### Fitting | ||
|
||
It is strongly recommended that if metacognitive efficiency is to be | ||
measured using the meta-d′/d′ method that researchers fist determine | ||
whether the Independent Truncated Gaussian Model, the confidence model | ||
implied by the meta-d′/d′ method, is an adequate description of the | ||
data. Using the function fitConfModel, we can fit several confidence | ||
models to the data of each participant. The argument | ||
`.parallel=TRUE`allows for parallelization over all but one available | ||
core. | ||
|
||
fitted_pars <- fitConfModels(MaskOri, models=c("SDT", "WEV"), .parallel = TRUE) | ||
|
||
This parallelizes the fitting process over participant-model | ||
combinations. The output is then a data frame with one row for each | ||
participant-model combination and columns for parameters and measures | ||
for model performance (negative log-likelihood, BIC, AIC and AICc). | ||
These may be used for quantitative model comparison. | ||
|
||
``` r | ||
head(fitted_pars) | ||
``` | ||
|
||
## model participant negLogLik N k BIC AICc AIC d_1 d_2 | ||
## 1 SDT 1 2721.256 1620 14 5545.975 5470.739 5470.513 0.0428 0.4593 | ||
## 2 WEV 1 2621.110 1620 16 5360.464 5274.520 5274.221 0.2027 0.6142 | ||
## 3 SDT 2 1946.258 1620 14 3995.979 3920.743 3920.517 0.0000 0.0950 | ||
## 4 WEV 2 1827.221 1620 16 3772.684 3686.741 3686.441 0.0512 0.1920 | ||
## 5 SDT 3 1706.178 1620 14 3515.818 3440.582 3440.356 0.2708 0.4673 | ||
## 6 WEV 3 1661.617 1620 16 3441.476 3355.533 3355.233 0.4146 0.8561 | ||
## d_3 d_4 d_5 c theta_minus.4 theta_minus.3 theta_minus.2 | ||
## 1 1.0526 3.6806 4.7779 -0.2723 -1.5467 -1.0333 -0.6336 | ||
## 2 1.0797 3.4746 4.0799 -0.2957 -2.0665 -1.2485 -0.4152 | ||
## 3 0.8601 6.1410 8.0556 -0.1394 -2.0092 -1.9193 -1.4097 | ||
## 4 1.0412 4.1423 5.2886 -0.1475 -2.0441 -1.9500 -1.3982 | ||
## 5 1.9117 6.4257 7.5755 -1.1510 -1.9938 -1.6372 -1.2600 | ||
## 6 2.7115 6.9164 7.9863 -1.3743 -2.7625 -1.9192 -0.3724 | ||
## theta_minus.1 theta_plus.1 theta_plus.2 theta_plus.3 theta_plus.4 sigma | ||
## 1 -0.4543 -0.0944 0.2152 0.9850 1.5735 NA | ||
## 2 0.1296 -0.6196 0.1544 1.3976 2.1879 1.0105 | ||
## 3 -0.9580 0.7857 1.3781 2.0879 2.2369 NA | ||
## 4 -0.9030 0.8201 1.4484 2.2447 2.4030 0.6391 | ||
## 5 -1.1668 -1.1143 -0.7344 0.2961 0.9314 NA | ||
## 6 0.9328 -2.7695 -1.1313 0.7714 1.7520 1.3289 | ||
## w wAIC wAICc wBIC | ||
## 1 NA 0 0 0 | ||
## 2 0.5361 1 1 1 | ||
## 3 NA 0 0 0 | ||
## 4 0.5020 1 1 1 | ||
## 5 NA 0 0 0 | ||
## 6 0.3818 1 1 1 | ||
|
||
If the Independent Truncated Gaussian model provides a decent account of | ||
the data (which is not the case though in the demo dataset), it is | ||
legitimate to quantify metacognitive efficiency with meta-d′/d′: | ||
|
||
MetaDs <- fitMetaDprime(subset(MaskOri, diffCond == "33.3"), | ||
model="ML", .parallel = TRUE) | ||
|
||
## Contact | ||
|
||
For comments, remarks, and questions please contact either | ||
<[email protected]> or <[email protected]> | ||
or [submit an issue](https://github.com/ManuelRausch/StatConfR/issues). | ||
|
||
# Acknowledgements | ||
|
||
This research was in part supported by grants RA2988/3-1 and RA2988/4-1 by the Deutsche Forschungsgemeinschaft. The funders had no role in software design, decision to publish, or preparation of the manuscript. Author contributions: Manuel Rausch: Conceptualization, Data curation, Funding acquisition, Software, Validation, Writing - original draft. Sebastian Hellmann: Conceptualization, Software, Writing - review and editing. | ||
|