'Project: When Language Matters. Study: Replicate S4B with two more conditions.' (AsPredicted #142972)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has one author.
Pre-registered on 2023/09/07 - 09:59 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Does the sequence of warm and competent language identified in our field data modeling (warm-competent-warm) outperform seven alternative sequences when it comes to customer satisfaction?
3) Describe the key dependent variable(s) specifying how they will be measured. How satisfied would you be with the agent? (1 = not at all satisfied, 7 = very much satisfied). We also attempt to replicate a helpfulness measure collected in field data Study 1 (How helpful was the agent? (1 = not at all helpful, 7 = very helpful).
4) How many and which conditions will participants be assigned to? Participants will be randomly assigned to one of eight conditions in a between-subjects design. The conditions differ in their emphasis on warm (affective) or competent (cognitive) language across the agents language over time, where time is divided into approximately the first 25%, middle 50%, and last 25% of the conversation (by word count). This study uses a short, carefully controlled design.
Our manipulations use words from LIWC's 2015 affective processes dictionary.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. Our primary statistical test of interest is a one-way analysis of variance (ANOVA) assessing the predicted simple effect of our dynamic treatment (warm-competent-warm) versus seven alternative sequences on the key dependent variable described above.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who move through the main stimuli page (the conversation transcript and dependent variable measure) at a time interval consistent with 500 WPM or greater (based on the number of words on the main stimuli page) according to a Qualtrics timer that is not observable to participants. The 500 WPM exclusion rule has been used by the first author in all laboratory studies conducted over the last 13 years. It is based on published guidelines on average adult reading speed and comprehension. Normal adult reading rates for comprehension are 200-250 WPM. 500 WPM captures more than three standard deviations (99.7%) of adult readers in the general population (Just & Carpenter, 1987; The Psychology of Reading and Language Comprehension).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will aim to have a final sample size of N = 75 per cell after accounting for exclusions. Based on prior studies using the same exclusion criterion, we expect an exclusion of approximately 5-10% of participants. We will therefore ask the panel to provide 5% more than our target to attempt to achieve our sample size goal.
Given the two conditions in the study, we will ask for 8 * 75 * 1.05 = 630 collected. If the final sample size is slightly smaller than 75 observations per condition (e.g. around 70 per condition), we will still proceed with the planned analysis. If any condition falls below 65 per condition after the exclusion, we will add 5 participants to each conditions until a minimum of 65 participants have completed each condition.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) None.
