'Study 1 (distinctiveness) replication – July 2021' (AsPredicted #71,793)
Author(s) Andrei Cimpian (New York University) - andrei.cimpian@nyu.edu Lin Bian (University of Chicago) - linbian@uchicago.edu
Pre-registered on 2021/07/30 09:44 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Hypotheses:
1) Category and individual generics will be endorsed more often for distinctive (vs. non-distinctive) behaviors, particularly among low-prevalence behaviors.
2) The facilitative effect of distinctiveness will not differ for generics about categories vs. individuals.
3) Describe the key dependent variable(s) specifying how they will be measured. Endorsement of a generic statement (true/false)
4) How many and which conditions will participants be assigned to? The design is as follows: 2 (type of generic: category vs. individual; between subjects) × 2 (distinctiveness: distinctive vs. non-distinctive; within subject) × 6 (level of prevalence: 1%, 5%, 10%, 30%, 50%, and 70%; within subject).
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will conduct a 2 (type of generic: category vs. individual; between subjects) × 2 (distinctiveness: distinctive vs. non-distinctive; within subject) × 6 (level of prevalence: 1%, 5%, 10%, 30%, 50%, and 70%; within subject) ANOVA.
To test Hypothesis 1 above, we will examine main effect of distinctiveness and the two-way interaction between distinctiveness and prevalence. We expect one or both of these to be significant. If significant, the interaction will be followed up with tests comparing endorsement of distinctive vs. non-distinctive behaviors at each level of prevalence.
To test Hypothesis 2 above, we will examine the two-way interaction between type of generic (category vs. individual) and distinctiveness and the three-way interaction between these two factors and prevalence. We expect neither of these to be significant.
As a further test of Hypothesis 2, we will calculate a Bayes Factor (BF01) to quantify the amount of support for the null hypothesis (H0: the effect of distinctiveness is the same for category and individual generics) over the alternative hypothesis (H1: the effect of distinctiveness is different for category and individual generics). To calculate BF01, we will first calculate a difference score for each participant by taking the difference between the proportion of "true" responses to distinctive items and the proportion of "true" responses to non-distinctive items, overall and at each prevalence level where we saw an effect of distinctiveness (see above). We will then conduct a Bayesian independent-samples t test that compares these difference scores for category vs. individual generics using JASP. We expect BF01 to be greater than 3 for the overall scores and each prevalence level tested.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude any participants (1) whose IP addresses are from outside the US or (2) who fail an attention check ("Please think back to the task you just completed. Did the questions ask about single individuals or entire species?").
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. 300 participants after exclusions. This is a ~50% increase in sample size relative to a study that we are directly replicating.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will test (and may describe in the write-up, when significant) the other main effects and interactions in the ANOVA.
In addition to (or instead of) ANOVAs, we may run mixed-effects logistic regressions with the same fixed-effects structure as the ANOVA above (1 between-subjects factor and 2 within-subjects factors) and with crossed random intercepts for participants and items.
These data will be included in a mini meta-analysis across several studies on the same topic.
