Inter-Item Differences in Metacognitive Judgments: Insights into the Collective Wisdomware Underlying These Judgments

Previous research introduced the concept of collective wisdomware within the frame-work of the self-consistency model of subjective confidence. According to this model, confidence in a binary choice arises from the random sampling of cues from a collective pool that is shared by individuals with similar backgrounds. Consequently, confidence should increase with the degree of agreement among respondents—a consensuality effect—and should predict the likelihood that other respondents will make the same choice—a replicability effect. This framework has recently been extended to judgments of learning (JOLs) and feelings of knowing (FOKs), and both types of judgments have likewise been shown to exhibit consensuality and replicability effects. In the present study, we report findings based on a reanalysis of previous data for confidence judgments, JOLs, and FOKs, indicating that the assumption of a uniformly shared collective wisdomware does not hold equally across all items. We consolidated data from several samples, including university students and school children, who completed computerized memory and perceptual tasks. Metacognitive judgments (confidence, FOK, JOL) were collected and analyzed for consensuality and replicability across and within participants. For each type of judgment, we identify reliable inter-item differences in the magnitude of the consensuality and replicability effects and examine several correlates of these item-level variations. Split-half reliability analyses confirm that this variance is a stable property of the items. The goal is to shed light on the nature of the collective wisdomware underlying each form of metacognitive judgment, as well as across these judgments. Our findings demonstrate that reliance on shared cues depends heavily on specific item characteristics.