'Examining the Inherence Bias in Practicing Scientists' (AsPredicted #167,860)
Author(s) Zachary Horne (University of Edinburgh) - Zachary.Horne@ed.ac.uk Kendall Smith (University of Edinburgh) - s2100690@ed.ac.uk Ana Ma (University of Edinburgh) - s2030310@ed.ac.uk Runyi Yao (University of Edinburgh) - s2044346@ed.ac.uk Andrei Cimpian (New York University) - andrei.cimpian@nyu.edu
Pre-registered on 2024/03/25 08:39 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? We are investigating whether practicing scientists from the University of Edinburgh rely on inherent facts when generating explanations of unfamiliar scientific phenomena (in this study, biological, chemical, and astrophysical phenomena).
3) Describe the key dependent variable(s) specifying how they will be measured. Participants are asked to provide open ended explanations to three scientific phenomena (one of each domain previously described). We will measure whether their explanation focuses on inherent facts in the observation (e.g., the chemicals compositions) and or/ whether their explanation focuses on extrinsic facts in the observation (e.g., the size/shape of the vessel the chemical reaction took place in).
4) How many and which conditions will participants be assigned to? This is a purely descriptive study and thus we do not rely on any manipulations beyond randomly assigning participants to receive some vignettes vs others. Participants receive three vignettes describing observations of phenomena out of a total of six vignettes (two were created for each domain). The grouping as well as the questions they received is randomized
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will fit a regression model to assess our main predictions.
We will firstly compare the frequency of inherent and extrinsic open-ended explanations by fitting a Bayesian logistic mixed-effects model with a weakly regularizing prior on two predictors and their interaction: type of explanation (Inherent vs. Extrinsic) and domain (chemistry, biology, physics). Our main prediction is we expect to see that inherent explanations are more common across all three domains, though the impact of domain on this inherence bias is unclear to us.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who indicated they did not pay attention -- question asks if participants took the study seriously and paid attention.
Participants who say they did not pay attention will be excluded from the analysis but will still be compensated.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will invite via email all researcher (either working towards or achieved a PhD) in all scientific departments (including social sciences), who have their email addresses publicly available on their department's website. We will not recruit from the psychology department, however.
The nature of our recruitment process mean that we are unsure of exactly how many participants will agree to participate - participant are emailed asking if they are willing to participate but response rates to these sorts of studies can be quite low.
We aim to recruit >= 50 participants - but, again, our recruitment procedures preclude us from being able to say with certainty exactly how many participants will be in the study once data collection is complete. (i.e., 1 week after they receive a reminder email if a given participant hasn't already completed the study).
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will explore possible relationships between domain of expertise (e.g., chemistry) and the inherence bias or open-ended explanations, though we have no specific predictions about this possible relationship.
