Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has 2 authors.
Pre-registered on 2021/11/17 04:53 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Participants will read two articles allegedly written by artificial intelligence (AI) based algorithms in one of three experimental conditions. The conditions will differ with respect to the between-group factor AI priming: Participants will either receive information about the strengths of AI in writing science journalistic articles or the weaknesses of AI in writing science journalistic articles or no such information (strengths vs. weaknesses vs. control). Additionally, participants will be presented with two articles about the same topic (climate change), which will differ with respect to the information presentation (within-group factor: neutral vs. evaluative) and will be presented in random order.
H1) There will be a main effect of AI priming on the perceived source credibility: The AI will be perceived to be more credible when its strengths are primed than when its weaknesses are primed.
H2) There will be a main effect of information presentation on the perceived source credibility: The AI will be perceived to be more credible for the article with a neutral information presentation than for the article with an evaluative information presentation.
H3) There will be an interaction effect of AI priming and information presentation on the perceived source credibility: The AI will be perceived to be most credible when its strengths are primed, and the information is presented neutrally.
H4) There will be a main effect of AI priming on the perceived message credibility: The articles will be perceived to be more credible when the AI's strengths are primed than when its weaknesses are primed.
H5) There will be a main effect of information presentation on the perceived message credibility: The article with a neutral information presentation will be perceived to be more credible than the article with an evaluative information presentation.
H6) There will be an interaction effect of AI priming and information presentation on the perceived message credibility: The article will be perceived to be most credible when the AI's strengths are primed, and the information is presented neutrally.
H7) There will be a main effect of information presentation on the perceived anthropomorphism of the AI: The AI will be perceived to be more anthropomorphic when the information is presented evaluatively than when presented neutrally.
3) Describe the key dependent variable(s) specifying how they will be measured. - Perceived credibility: credibility, both concerning the AI (source credibility) and concerning the article (message credibility), will be measured on five bipolar items (Flanagin & Metzger, 2000, 2007; 7-point Likert scales).
- Perceived anthropomorphism and intelligence of the AI: five bipolar items per respective subscale (Bartneck, Kulić, Croft & Zoghbi, 2009) measured on 5-point Likert scales.
4) How many and which conditions will participants be assigned to? Participants will be randomly assigned to one of three conditions resulting from the between-factor AI priming: AI strengths, AI weaknesses, control (no priming).
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will conduct mixed design ANOVAs with the between-group factor AI priming and the within-group factor information presentation as repeated measures. We will conduct three ANOVAs, one each for source credibility, message credibility, and perceived anthropomorphism.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who incorrectly answer the manipulation check for the factor AI priming or the attention check (related to the articles' contents and the authorship).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. A power analysis for a small effect size of f = 0.1, an alpha-error probability of 0.05, a power of 0.80, and an estimated correlation among repeated measures of 0.30 revealed a total sample size of 342 participants. As we suspect that we will have to exclude some participants due to incorrect answers to the manipulation check or the attention check, we will collect observations from 380 participants.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) For control purposes concerning the information presentation manipulation, we will measure the perceived neutrality of each text by one Likert-type item on a five-point scale ranging from "absolutely neutral" to "absolutely evaluative".
Concerning the perceived intelligence, we state an open research question:
RQ1: Do AI priming or information presentation influence the perceived intelligence of the AI?
Further measures for exploratory purposes:
- Self-reported familiarity with automated text-generation, self-reported knowledge about AI, and scepticism toward AI will be measured on single 5-point Likert items.
- Participants' plausibility perception concerning a human-induced climate change. Following the Plausibility Perceptions Measure (Lombardi & Sinatra, 2012; Lombardi et al., 2013), we adapt a single item to ask the participants how plausible they consider climate change to be human induced (10-point Likert scale).
RQ2: Does the inclusion of the participants' plausibility perceptions as a control variable influence the effects of AI priming and information presentation on the key dependent variables?
- Participants' belief in the machine heuristic: Four items adapted from Waddell (2018) and based on a conceptual definition offered by the MAIN model (Sundar, 2008) to measure the belief in the machine heuristic (on a 5-point Likert scale).
RQ3: Is there any effect of AI priming and information presentation on the belief in the machine heuristic?
- Participants' intentions to read such AI written articles again in the future, their intention to consult an AI on this subject again, and their intention to recommend it to a person close to them (friend/family member) on 5-point Likert scale items.
RQ4: Is there any effect of AI priming and information presentation on the intention to read AI written articles again, to consult an AI on this subject, or to recommend it to a person close to them?
