New md

README.rmd

@@ -20,7 +20,9 @@ gitbranch <- "main/"
 
 The `statConfR` package provides functions to fit static models of
 decision-making and confidence derived from signal detection theory for
-binary discrimination tasks, meta-d′/d′, a wide-spread measure of metacognitive efficiency, as well as information-theoretic measures of metacognitive sensitivity and efficiency. 
+binary discrimination tasks, meta-d′/d′, a wide-spread measure of metacognitive efficiency, 
+meta-I, an information-theoretic measures of metacognitive sensitivity, 
+as well as $meta-I_{1}^{r}$ and $meta-I_{2}^{r}$, two information-theoretic measures of metacognitive efficiency. 
 
 Fitting models of confidence can be used to test the assumptions underlying
 meta-d′/d′. Several static models of decision-making and confidence include a metacognition parameter that may
@@ -162,14 +164,15 @@ because both are measured on the same scale. Meta-d′ can be compared against t
 ### Information-theoretic measures of metacognition
 
 Dayan (2023) proposed several measures of metacognition based on quantities of information theory. 
-* Meta-I is a measure of metacognitive sensitivity defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the accuracy,
-$$meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y})$$. 
+
+- Meta-I is a measure of metacognitive sensitivity defined as the mutual information between the confidence and accuracy and is calculated as the transmitted information minus the minimal information given the accuracy,
+$$meta-I = I(Y; \hat{Y}, C) - I(Y; \hat{Y}).$$ 
 This is equivalent to Dayan's formulation where meta-I is the information that confidences transmit about the correctness of a response: 
 $$meta-I = I(Y = \hat{Y}; C)$$
-* Meta-$I_{1}^{r}$ is meta-I  normalized by the value of meta-I expected assuming 
+- Meta-$I_{1}^{r}$ is meta-I  normalized by the value of meta-I expected assuming 
 a signal detection model (Green & Swets, 1966) with Gaussian noise, based on calculating the sensitivity index d':
 $$meta-I_{1}^{r} = meta-I / meta-I(d')$$
- * Meta-$I_{2}^{r}$ is meta-I normalized by its theoretical upper bound, which is the information entropy of accuracy,  $H(Y = \hat{Y})$:
+- Meta-$I_{2}^{r}$ is meta-I normalized by its theoretical upper bound, which is the information entropy of accuracy,  $H(Y = \hat{Y})$:
 $$meta-I_{2}^{r} = meta-I / H(Y = \hat{Y})$$
 
 Notably, Dayan (2023) pointed out that a liberal or conservative use of the confidence levels will affected the mutual information and thus all information-theoretic measures of metacognition. 
@@ -209,7 +212,8 @@ head(MaskOri)
 The function `fitConfModels` allows the user to fit several confidence
 models separately to the data of each participant. 
 The data should be provided via the argument `.data` in the form of a data.frame 
-object with the following variables in separate columns: 
+object with the following variables in separate columns:
+
 -   stimulus (factor with 2 levels): The property of the stimulus which
     defines which response is correct
 -   diffCond (factor): The experimental manipulation that is expected to
@@ -240,7 +244,7 @@ It can be seen that the independent truncated Gaussian model is consistently out
 
 ### Visualization 
 
-After obtaining model fits, it is strongly recommended to visualize the prediction implied by the best fitting sets of parameters and to compare the prediction with the actual data. The best way to visualize the data is highly specific to the data set and research question, which is why `statConfR` does not come with its own visualization tools. This being said, here is an example for how a visualization could look like: 
+After obtaining model fits, it is strongly recommended to visualize the prediction implied by the best fitting sets of parameters and to compare the prediction with the actual data (Palminteri et al., 2017). The best way to visualize the data is highly specific to the data set and research question, which is why `statConfR` does not come with its own visualization tools. This being said, here is an example for how a visualization of model fit could look like: 
 
 <!-- Stuff where only the code should be shown and executed, but do not show R yapping  -->
 
@@ -304,6 +308,7 @@ MetaDs <- fitMetaDprime(data = MaskOri, model="ML", .parallel = TRUE)
 ```
 
 To estimate information theoretic measures of metacognition, we must bring first the data into the correct format. Specifically, three different types of inputs are accepted:
+
 * A `data.frame` with variables "y" for true labels and "r" for confidence-binned responses. "y" needs to contain values -1 and +1 while r needs to be a factor with ordered levels such that the first half of the levels correspond to predictions for y = -1 and the second half to predictions for y = +1.
 * A counts `table` with joint absolute frequencies. Rows correspond to true labels (stimulus categories) and columns correspond to responses.
 * A contingency `matrix` with joint relative frequencies (as before but normalized to sum up to 1).
@@ -363,6 +368,7 @@ or [submit an issue](https://github.com/ManuelRausch/StatConfR/issues).
 - Maniscalco, B., & Lau, H. (2012). A signal detection theoretic method for estimating metacognitive sensitivity from confidence ratings. Consciousness and Cognition, 21(1), 422–430. https://doi.org/10.1016/j.concog.2011.09.021
 - Maniscalco, B., & Lau, H. (2016). The signal processing architecture underlying subjective reports of sensory awareness. Neuroscience of Consciousness, 1, 1–17. https://doi.org/10.1093/nc/niw002
 - Maniscalco, B., & Lau, H. C. (2014). Signal Detection Theory Analysis of Type 1 and Type 2 Data: Meta-d, Response- Specific Meta-d, and the Unequal Variance SDT Model. In S. M. Fleming & C. D. Frith (Eds.), The Cognitive Neuroscience of Metacognition (pp. 25–66). Springer. https://doi.org/10.1007/978-3-642-45190-4_3
+- Palminteri, S., Wyart, V., & Koechlin, E. (2017). The importance of falsification in computational cognitive modeling. Trends in Cognitive Sciences, 21(6), 425–433. https://doi.org/10.1016/j.tics.2017.03.011
 - Rausch, M., Hellmann, S., & Zehetleitner, M. (2018). Confidence in masked orientation judgments is informed by both evidence and visibility. Attention, Perception, and Psychophysics, 80(1), 134–154. https://doi.org/10.3758/s13414-017-1431-5
 - Rausch, M., & Zehetleitner, M. (2017). Should metacognition be measured by logistic regression? Consciousness and Cognition, 49, 291–312. https://doi.org/10.1016/j.concog.2017.02.007
 - Shekhar, M., & Rahnev, D. (2021). The Nature of Metacognitive Inefficiency in Perceptual Decision Making. Psychological Review, 128(1), 45–70. https://doi.org/10.1037/rev0000249

paper.md

@@ -52,10 +52,12 @@ binary discrimination tasks with confidence ratings:
 - the independent truncated Gaussian model [@rausch_measures_2023] based on the model specification 
 used for the original meta-d$^\prime$/d$^\prime$ method [@Maniscalco2012; @Maniscalco2014], and 
 - the independent truncated Gaussian model based on the model specification of Hmetad [@Fleming2017a]. 
-In addition, the `statConfR` package provides functions for estimating two different 
+In addition, the `statConfR` package provides functions for estimating different 
 kinds of measures of metacognition: 
 - meta-d$^\prime$/d$^\prime$, the most widely-used measure of metacognitive efficiency, allowing both @Maniscalco2012's and @Fleming2017a's model specification, 
-- Information-theoretic measures of metacognitive sensitivity and metacognitive efficiency [@dayan_metacognitive_2023]. 
+- Information-theoretic measures [@dayan_metacognitive_2023], including
+  - meta-I, an information-theoretic measures of metacognitive sensitivity, 
+  - $meta-I_{1}^{r}$ and $meta-I_{2}^{r}$, two measures of metacognitive efficiency. 
 Finally, the `statConfR` package includes an example data set previously published in @hellmann_simultaneous_2023, with which the functions can be  tested. 
 The `statConfR` reference manual provides documentation of each function of the latest release (https://cran.r-project.org/web/packages/statConfR/statConfR.pdf). 
 
@@ -92,7 +94,7 @@ The `statConfR` package also provides a faithful implementation of meta-d$^\prim
 which has been originally implemented in MATLAB [@Maniscalco2012]. Fleming provides MATLAB and R code for Hmetad, 
 a Bayesian hierarchical version of meta-d$^\prime$/d$^\prime$ [@Fleming2017a], 
 but notably he specifies the model slightly differently as in the original meta-d$^\prime$/d$^\prime$ [@rausch_measures_2023]. 
-To our knowledge, information-theoretic
+To our knowledge, there has been no open software available to estimate information-theoretic measures of metacognition up to now. 
 
 An important limitation of the models implemented in `statConfR` is that the dynamics of the decision process are not taken into account.
 This is a problem because confidence judgments are related to the dynamics of decision making [@hellmann_confidence_2024; @Pleskac2010; @Rahnev2020].