'When Lang Matters Replication versus Competent-Warm-Competent Control' (AsPredicted #142957)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has one author.
Pre-registered on 2023/09/07 - 08:37 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Rather than an alternative sequence that uses the same overall count and proportion of warm (affective) words (per LIWC) but at the "wrong" time, could agent bookending of customer service conversations with warm language at the start and end improve customer satisfaction?
3) Describe the key dependent variable(s) specifying how they will be measured. How satisfied would you be with the agent? (1 = not at all satisfied, 7 = very much satisfied). We also attempt to replicate a helpfulness measure collected in field data Study 1 (How helpful was the agent? (1 = not at all helpful, 7 = very helpful).
4) How many and which conditions will participants be assigned to? Participants will be randomly assigned to one of two conditions in a between-subjects design. The conditions differ in their emphasis on warm (affective) or competent (cognitive) language across the agents language over time, where time is divided into approximately the first 25%, middle 50%, and last 25% of the conversation (by word count). The first condition is the treament that presents our recommended dynamic sequence over these time periods (warm-comp-warm). The second condition is intended to act as a more "conservative" control than the "competent throughout" baseline control used in earlier studies by carefully ensuring that the agent's use of warm words is exactly the same (as a proportion of total words) as the treatment condition, but occuring at the "wrong" time (comp-warm-comp) as recommended by a member of the review team.
All participants will be presented with the turn-by-turn transcript of an airline customer service conversation. Our manipulations use words from LIWC's 2015 affective processes dictionary, prioritizing words that are also shared by Marinova, Singh and Singh's (2018 JMR) relating dictionary.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. Our primary statistical test of interest is a one-way analysis of variance (ANOVA) assessing the predicted simple effect of treatment versus the control on the key dependent variables described above.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who move through the main stimuli page (the conversation transcript and dependent variable measure) at a time interval consistent with 500 WPM or greater (based on the number of words on the main stimuli page) according to a Qualtrics timer that is not observable to participants. The 500 WPM exclusion rule has been used by the first author in all laboratory studies conducted over the last 13 years. It is based on published guidelines on average adult reading speed and comprehension. Normal adult reading rates for comprehension are 200-250 WPM. 500 WPM captures more than three standard deviations (99.7%) of adult readers in the general population (Just & Carpenter, 1987; The Psychology of Reading and Language Comprehension).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will aim to have a final sample size of N = 75 per cell after accounting for exclusions. Based on prior studies using the same exclusion criterion, we expect an exclusion of approximately 5-10% of participants. We will therefore ask the panel to provide 5% more than our target to attempt to achieve our sample size goal.
Given the two conditions in the study, we will ask for 2 * 75 * 1.05 = 158 collected. If the final sample size is slightly smaller than 75 observations per condition (e.g. 70 per condition or more), we will still proceed with the planned analysis. If either condition falls below 70 per condition after the exclusion, we will add 10 participants to each condition until a minimum of 70 participants have completed each condition.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will also collect the perceived warmth and competence mediators collected in a prior study ("How warm was the agent?", "How competent was the agent?", 1 = not at all, 7 = very much) to further assess process in this study.
