'Marginal probabilities of motion verb scenarios' (AsPredicted #17471)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has one author.
Pre-registered on 12/05/2018 08:24 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? This experiment tests a prediction made by our Perspectival Rational Speech Acts model about the likelihood of worlds according to listeners of sentences containing perspectival items once the perspectives are marginalized out. We test whether, all else being equal, listeners prefer to interpret perspectival sentences like "Sarah is coming to the library" as describing a scenario where both the speaker and listener are located at the library; or whether they consider all scenarios where the speaker is located at the library (regardless of the listener's location) as equally likely.
We plan to measure this in decision task where participants evaluate sentences in contexts (which represent possible worlds). Participants will be asked to imagine that they are Lucy (a clip-art image) and have a friend named Sam (another image). Participants will see an image of Sam with a speech bubble containing a sentence. After a brief pause, they will see Sam speaking the sentence in an illustrated context, which also shows Lucy's location. They will be asked to choose whether what Sam says makes sense in the context or not (binary choice). We will also measure reaction times.
The sentences will either contain the motion verb "come" or a non-perspectival motion verb ("walk", "drive"), which controls for general preferences for the characters to be located together or separately. There are 8 conditions, created by varying the sentence (perspectival v. plain) with 4 images. The images depict Sam and Lucy together at the destination of motion; apart, with Sam at the destination; apart, with Lucy at the destination; and together, with neither of them at the destination.
Because most of the main items are true, our fillers are biased towards false.
We also have 5 sets of 4 spatial items interspersed at 25% breakpoints in the main items, which test how well the participants are able to identify with the spatial orientation of the Lucy character. The design of these items is similar to the main items, except that the sentences test spatial descriptions rather than perspectival motion verbs ("The chair closest to you is yellow"; "The car between you and the streetlight is blue").
3) Describe the key dependent variable(s) specifying how they will be measured. Two measures: reaction time (ms taken to select YES or NO on the image page) and accuracy (rate of choosing YES to accept sentences that fit the pictures and NO to reject sentences that do not fit the pictures). Participants will be timed out from responding after 10 seconds.
4) How many and which conditions will participants be assigned to? Participants will see 6 items in each of the 8 conditions, presented using a Latin Square design. The spatial items will occur before and after the experiment, as well as at 3 intermediate points breaking up the main items.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will use mixed-effects modeling on the reaction time data to test the effect of the perspectival condition on the relative rates of acceptance of the perspectival sentences matched with the images where both Lucy and Sam are at the destination of motion.
We will compare reaction time data from observations where participants answered YES. We will also log-transform the data, since RT data is not usually normally distributed.
Then we will fit two mixed-effects models to the data: (1) a model that includes both the fixed effects of perspective and scenario along with the the random participant and item effects; and (2) a model that includes the interaction of perspective and scenario along as well as all effects in (1). We will do a likelihood ratio test between 1 and 2 to establish whether there is a significant interaction between scenario and perspective.
If there is, we will look at a handful of pairwise comparisons: (1) between the speaker scenario and the speaker+addressee scenario; (2) between the speaker scenario and the listener scenario; and (3) between the speaker+addressee scenario and the listener scenario. We will look at the difference of least squares means for these comparisons.
Our hypothesis predicts a significant difference for (1); while the alternative speaker-default hypothesis predicts no significant difference for (1), but a significant difference for (2) and (3).
We expect these pairwise comparisons to be significant in the perspectival condition, but not the plain condition.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants who fail to achieve 90% accuracy on the spatial items that do not involve right/left. 5 of the spatial items involve decisions about left and right, which are very challenging for some individuals. We will not use these as exclusion criteria.
We will also exclude participants who indicate that their native language is not English, or who were exposed to significant amounts of a different language as a child.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will collect data from 80 participants.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will also collect demographic information from participants for exploratory purposes. We may do exploratory analysis of the spatial items to understand which kinds are hardest for participants (right/left versus in front/behind, etc.)
