1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Hypotheses for change probability
H1: Experts change presented judgments more frequently than novices.
H2: Judgments from novices are changed more frequently.
H3: More accurate judgments are less likely to be changed than less accurate judgments.
Hypotheses for Improvement
H1: Experts improve presented judgments more than novices.
H2: Presented judgments of novices are improved more than presented judgments of experts.
H3: More accurate judgments are improved less than less accurate judgments.
3) Describe the key dependent variable(s) specifying how they will be measured. Change Probability: Probability of a participant to make a change to a presented judgment of a previous participant
Improvement: Relative difference between the accuracy of the presented judgment and the accuraty of the provided judgment. Accuracy is computed as the absolute relative error of the provided judgment or the presented judgment, respectively.
4) How many and which conditions will participants be assigned to? Manipulation of own expertise (between-subjects)
To manipulate participants' expertise, we will use the random-dots estimation task (Mayer et al., 2023).
Experts learn to approximate the number of dots by laying a raster over the image, counting the dots in one box and multiplying by nine
Novices read an essay not helping with making more accurate judgments
Manipulation of others' expertise (within-subjects)
The participants receive information about the expertise (high vs. low vs. no information) of the previous person providing the judgment.
Quality of the judgment (within-subjects)
The presented judgment of the previous person differs either +/- 70% (low quality), +/- 35% (medium quality) or 0% (high quality) from the correct value.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will use generalized linear mixed models to analyze hypotheses concerning whether participants change a given judgment (i.e., dichotomous values of 0 - no change and 1 - change) and linear mixed models to predict the improvement of judgments. We account for the nested structure of our data with random intercepts for participants and items.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. Excluding participants
Participants are excluded during participation if they change the browser window against the instructions more than 5 times. Participants are excluded if they provide judgments that depict low effort or no interest in the study, which include providing the same answer, number sequences or extreme judgments with more than 150% deviation for more than 20% of items.
Excluding judgments
Based on a pliot study (Mayer et al., 2023), participants cannot enter values larger than 2000 as their judgment. Moreover, we exclude judgments that are timed-out after 60 seconds. Furthermore, participants are already excluded during participation if they do not consent to the terms and conditions of the study, access the study with mobile devices, change the browser window more than 5 times during participation, or answer less than 3 of 4 control questions correctly after being instructed to use raster scanning.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will at least sample 72 participants. This sample size was derived from a power analysis for the between subjects effect with a power of 1-β = .80, alpha = .05, and a medium effect size of d = .25.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) Exploratively we will analyze the judgments of the participants encountering judgments for which no information about the previous participant is available. We assume that change probabilities and improvement will fall in between the ones observed for previous participants who are labeled as experts or novices.
Moreover, we examine interaction effects between participants' expertise, expertise of the previous participant, and accuracy of the previous judgment.
