'Naturalistic Test of When Language Matters Accounting for Stimulus Sampling' (AsPredicted #125146)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has one author.
Pre-registered on 2023/03/14 - 04:03 PM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Does using more warm language at the start and end of customer service conversations enhance customer satisfaction?
3) Describe the key dependent variable(s) specifying how they will be measured. How satisfied would you be with the employee? (1 = not at all satisfied, 7 = very much satisfied)
4) How many and which conditions will participants be assigned to? Participants will be randomly assigned to one of 10 conditions in a 2(control vs. warm start and end treatment) x 5(stimuli sampling) between-subjects design.
All participants will be presented with the turn-by-turn transcript of a real customer service conversation. Any customer personally identifying information will be redacted (e.g., name, email address, mailing address). The control condition will be the actual conversation after any redactions. The treatment condition will make changes to the employee's language in the first and last 25% of the conversation to increase their use of affective language. We use words from LIWC's affective processes dictionary, prioritizing those words that are also shared by Marinova, Singh and Singh's (2018 JMR) relating dictionary. We only manipulate affective language linked to warmth at the start and end because prioritizing competence throughout a customer service conversation is already the primary recommendation of prior research, and as such, our field data findings that cognitive language should be used in the middle 50% of the conversation is consistent with prior research.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. Our primary statistical test of interest is a single one-way analysis of variance (ANOVA) assessing the predicted main effect of treatment across the full sample of five real conversations. If a statistically significant result is observed, a planned follow-up test will add conversation stimuli (5 samples) as a control factor, and report any simple effects and interaction of this factor with the focal treatment effect.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who move through the main stimuli page (the conversation transcript and dependent variable measure) at a time interval consistent with 500 WPM or greater (based on the number of words on the main stimuli page) according to a Qualtrics timer that is not observable to participants. The 500 WPM exclusion rule has been used by the first author in all laboratory studies conducted over the last 13 years. It is based on published guidelines on average adult reading speed and comprehension. Normal adult reading rates for comprehension are 200-250 WPM. 500 WPM captures more than three standard deviations (99.7%) of adult readers in the general population (Just & Carpenter, 1987; The Psychology of Reading and Language Comprehension).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We aim to have a final sample size of N = 75 per cell after accounting for exclusions to replicate consistent with sample size target in the first controlled experiment for this project. Based on prior studies using the same exclusion criterion, we expect an exclusion of approximately 5-10% of participants. We will therefore ask the panel to provide 5% more than our target to attempt to achieve our sample size goal.
Given the 10 conditions in the study, we will ask for 750 x 1.05 = 788 collected. If the final sample size is slightly smaller than 75 observations per condition, we will still proceed with the planned analysis.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) None.
