diff --git a/README.md b/README.md index 524b970..6aedc96 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ The `statConfR` package provides functions to fit static models of decision-making and confidence derived from signal detection theory for -binary discrimination tasks, meta-d′/d′, a wide-spread measure of +binary discrimination tasks, meta-d′/d′, the most prominent measure of metacognitive efficiency, meta-I, an information-theoretic measures of metacognitive sensitivity, as well as $`meta-I_{1}^{r}`$ and $`meta-I_{2}^{r}`$, two information-theoretic measures of metacognitive @@ -210,28 +210,31 @@ quantities of information theory. - Meta-I is a measure of metacognitive sensitivity defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the - accuracy, - ``` math - meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y}). - ``` - This is equivalent to Dayan’s formulation where meta-I is the - information that confidences transmit about the correctness of a - response: - ``` math - meta-I = I(Y = \hat{Y}; C) - ``` -- Meta-$`I_{1}^{r}`$ is meta-I normalized by the value of meta-I - expected assuming a signal detection model (Green & Swets, 1966) with - Gaussian noise, based on calculating the sensitivity index d’: - ``` math - meta-I_{1}^{r} = meta-I / meta-I(d') - ``` -- Meta-$`I_{2}^{r}`$ is meta-I normalized by its theoretical upper - bound, which is the information entropy of accuracy, - $`H(Y = \hat{Y})`$: - ``` math - meta-I_{2}^{r} = meta-I / H(Y = \hat{Y}) - ``` + accuracy: + +``` math +meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y}) +``` +This is equivalent to Dayan’s formulation where meta-I is the +information that confidences transmit about the correctness of a +response: + +``` math +meta-I = I(Y = \hat{Y}; C) +``` + - Meta-$`I_{1}^{r}`$ is meta-I normalized by the value of meta-I +expected assuming a signal detection model (Green & Swets, 1966) with +Gaussian noise, based on calculating the sensitivity index d’: + +``` math +meta-I_{1}^{r} = meta-I / meta-I(d') +``` + - Meta-$`I_{2}^{r}`$ is meta-I normalized by its theoretical upper +bound, which is the information entropy of accuracy, $`H(Y = \hat{Y})`$: + +``` math +meta-I_{2}^{r} = meta-I / H(Y = \hat{Y}) +``` Notably, Dayan (2023) pointed out that a liberal or conservative use of the confidence levels will affected the mutual information and thus all @@ -410,7 +413,8 @@ PlotMeans <- aes(ymin = ratings-se, ymax = ratings+se), color="black") + geom_point(data = AggregatedData, aes(shape=correct), color="black") + scale_shape_manual(values = c(15, 16), - labels = c("Error", "Correct response"), name = "observed data") + labels = c("Error", "Correct response"), name = "observed data") + + theme_bw() ``` @@ -463,12 +467,13 @@ r <- factor(ifelse(OneSbj$response == 0, -1, 1) * as.numeric(OneSbj$rating)) counts <- table(y, r) ``` -Then, the different information-theoretic measures of metacognitive -sensitivity and accuracy can be computed: +Then, the different information-theoretic measures of metacognition can +be computed: ``` r +meta_I <- estimate_meta_I(counts) meta_Ir1 <- estimate_meta_Ir1(counts) -meta_Ir1 <- estimate_meta_Ir1(counts) +meta_Ir1_acc <- estimate_meta_Ir1_acc(counts) meta_Ir2 <- estimate_meta_Ir2(counts) RMI <- estimate_RMI(counts) ``` @@ -476,7 +481,7 @@ RMI <- estimate_RMI(counts) ### Documentation The documentation of each function of the currently installed version of -`statConfR` can be accessed by typing ?*functionname* into the console. +`statConfR` can be accessed by typing *?functionname* into the console. ## Contributing to the package @@ -531,6 +536,8 @@ issue](https://github.com/ManuelRausch/StatConfR/issues). ## References +- Dayan, P. (2023). Metacognitive Information Theory. Open Mind, 7, + 392–411. - Fleming, S. M. (2017). HMeta-d: Hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings. Neuroscience of Consciousness, 1, 1–14. diff --git a/README.rmd b/README.rmd index 9c85c05..4012309 100644 --- a/README.rmd +++ b/README.rmd @@ -20,7 +20,7 @@ gitbranch <- "main/" The `statConfR` package provides functions to fit static models of decision-making and confidence derived from signal detection theory for -binary discrimination tasks, meta-d′/d′, a wide-spread measure of metacognitive efficiency, +binary discrimination tasks, meta-d′/d′, the most prominent measure of metacognitive efficiency, meta-I, an information-theoretic measures of metacognitive sensitivity, as well as $meta-I_{1}^{r}$ and $meta-I_{2}^{r}$, two information-theoretic measures of metacognitive efficiency. @@ -165,14 +165,18 @@ because both are measured on the same scale. Meta-d′ can be compared against t Dayan (2023) proposed several measures of metacognition based on quantities of information theory. -- Meta-I is a measure of metacognitive sensitivity defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the accuracy, -$$meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y}).$$ +- Meta-I is a measure of metacognitive sensitivity defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the accuracy: + +$$meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y})$$ This is equivalent to Dayan's formulation where meta-I is the information that confidences transmit about the correctness of a response: + $$meta-I = I(Y = \hat{Y}; C)$$ - Meta-$I_{1}^{r}$ is meta-I normalized by the value of meta-I expected assuming a signal detection model (Green & Swets, 1966) with Gaussian noise, based on calculating the sensitivity index d': + $$meta-I_{1}^{r} = meta-I / meta-I(d')$$ - Meta-$I_{2}^{r}$ is meta-I normalized by its theoretical upper bound, which is the information entropy of accuracy, $H(Y = \hat{Y})$: + $$meta-I_{2}^{r} = meta-I / H(Y = \hat{Y})$$ Notably, Dayan (2023) pointed out that a liberal or conservative use of the confidence levels will affected the mutual information and thus all information-theoretic measures of metacognition. @@ -289,7 +293,8 @@ PlotMeans <- aes(ymin = ratings-se, ymax = ratings+se), color="black") + geom_point(data = AggregatedData, aes(shape=correct), color="black") + scale_shape_manual(values = c(15, 16), - labels = c("Error", "Correct response"), name = "observed data") + labels = c("Error", "Correct response"), name = "observed data") + + theme_bw() ``` @@ -320,18 +325,19 @@ r <- factor(ifelse(OneSbj$response == 0, -1, 1) * as.numeric(OneSbj$rating)) counts <- table(y, r) ``` -Then, the different information-theoretic measures of metacognitive sensitivity and accuracy can be computed: +Then, the different information-theoretic measures of metacognition can be computed: ```{r} +meta_I <- estimate_meta_I(counts) meta_Ir1 <- estimate_meta_Ir1(counts) -meta_Ir1 <- estimate_meta_Ir1(counts) +meta_Ir1_acc <- estimate_meta_Ir1_acc(counts) meta_Ir2 <- estimate_meta_Ir2(counts) RMI <- estimate_RMI(counts) ``` ### Documentation -The documentation of each function of the currently installed version of `statConfR` can be accessed by typing ?*functionname* into the console. +The documentation of each function of the currently installed version of `statConfR` can be accessed by typing *?functionname* into the console. ## Contributing to the package The package is under active development. We are planning to implement new models of decision confidence when they are published. Please feel free to [contact us](malto::manuel.rausch@ku.de) to suggest new models to implement in in the package, or to volunteer adding additional models. diff --git a/README_files/figure-gfm/unnamed-chunk-5-1.png b/README_files/figure-gfm/unnamed-chunk-5-1.png index 8cca74a..03c60ef 100644 Binary files a/README_files/figure-gfm/unnamed-chunk-5-1.png and b/README_files/figure-gfm/unnamed-chunk-5-1.png differ