'Online sentence-picture reactivation-validation experiment with successive order'
(AsPredicted #101,349)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
This study is related to the study "Online text-picture validation study" (#73263). We investigate whether sentence-picture combinations are reactivated and validated automatically. In contrast to study #73263, the sentence and the picture will be presented successively (i.e., sentence before picture, or vice versa)
Each sentence-picture combination can be valid (sentence describes the picture correctly) or invalid (sentence describes the picture incorrectly). The combinations are presented successively: First, participants see a sentence or a picture. Second, they have to respond to the probe word "wrong" by pressing the key D or to the probe word "right" by pressing the key K. Third, participants have to solve a simple arithmetic problem. Fourth, the picture or sentence that corresponds to the sentence or picture presented before is presented. After that, participants have to react again to the probe words "wrong" or "right". Please note that the hypotheses described below refer to the reaction time and error rate regarding this last probe.
We hypothesize that the reaction time after congruent stimuli (sentence-picture combination is valid, probe word is right or sentence-picture combination is invalid, probe word is wrong) is faster than after incongruent stimuli (sentence-picture combination is invalid, probe word is right or sentence-picture combination is valid, probe word is wrong). This should result in a significant interaction between validity and probe word meaning that for valid sentence-picture combinations faster reaction times regarding the reaction to the probe word "right" than to the probe word "wrong" are expected, whereas for invalid sentence-picture combinations faster reaction times for the probe word "wrong" than for the probe word "right" are expected. Looking at the interaction the other way around this means that for the probe word "right" the reaction time is slower after invalid items than after valid items, whereas for the probe word "wrong" the reaction time is slower after valid than after invalid items.
Moreover, we hypothesize that the error rate after congruent stimuli is lower than after incongruent stimuli. This should result in a significant interaction between validity and probe word, meaning that for valid sentence-picture combinations less errors regarding the reaction to the probe word "right" than to the probe word "wrong" are expected, whereas for invalid sentence-picture combinations less errors for the probe word "wrong" than for the probe word "right" are expected. Looking at the interaction the other way around this means that for the probe word "right" the error rates are higher after invalid items than after valid items, whereas for the probe word "wrong" the error rates are higher after valid than after invalid items.

3) Describe the key dependent variable(s) specifying how they will be measured.
Reaction time: Time until a participant presses the key D or K after seeing the probe word measured in milliseconds. Please note that in the analysis only the reaction time for the second probe word is included.
Error rate: How many times a participant presses the key D after seeing the probe word right and presses the key K after seeing the probe word wrong. Please note that in the analysis only the error rate for the second probe word is included.

4) How many and which conditions will participants be assigned to?
There are four within-subjects conditions resulting from crossing the factor sentence-picture validity (valid vs. invalid) and the factor probe word (right vs. wrong). Moreover, as an exploratory factor, we will include the factor order of presentation (picture first vs. sentence first).
We will use eight counterbalanced lists of items: Participants are randomly assigned to one of those eight lists. Every list contains 80 sentence-picture combinations. The number of valid and invalid sentence-picture combinations, the number of the probe words, and the number of congruent and incongruent trials are the same in all lists. Moreover, after 20 randomly selected trials, participants have to solve an attention check task and indicate whether they have just seen this sentence or this picture. The list will be presented in two blocks à 40 trials each.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
Reaction time: Linear mixed effects model with item and participant as random effects, probe word, and validity as fixed effects will be conducted. If the interaction of validity * probe word is significant, pairwise single comparisons will be conducted. We will compare a) reaction times for probe words (wrong vs. right) for invalid as well as valid stimuli, and b) reaction times for validity (valid vs. invalid) for the probe words right vs. wrong.

Error rate: General linear mixed effects model with item and participant as random effects, probe word, and validity as fixed effects. If the interaction of validity * probe word is significant, pairwise single comparisons will be conducted. We will compare a) error rates for probe words (wrong vs. right) for invalid as well as valid stimuli, and b) error rates for validity (valid vs. invalid) for the probe words right vs. wrong<

In case the interactions are not significant, we will run additional analyses to test whether the single comparisons reach significance.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
Exclusions of participants: Participants who do not agree to have their data processed will be deleted. Participants have to speak German fluently and be right-handed. Participants who report serious technical issues during the study will be excluded as well. Moreover, participants with more than 40% incorrect responses to the probe words (right and wrong) will be excluded. Participants with an error rate above 40% in the attention check tasks will be excluded as well.
Exclusion of trials: Trials with reaction times for the second probe word under 10ms or above 5s will be excluded. For reaction time analysis, trials with incorrect responses for the second probe word will be excluded.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
We conducted a power simulation based on the data of previous studies (#73263, and #89392). Validity and probe word are contrast coded (invalid = 1, valid = -1; wrong = 1, right = -1) and reaction time data is log transformed.
The simulations revealed that 250 participants lead to sufficient power. Hence, we plan to recruit participants until we have approximately 250 valid cases.
The parameters for the reaction time analysis were set as follows:
Fixed effects: estimate intercept = 6.508 estimate probe word = 0.006, estimate validity = 0.009, estimate probe word * Validity = -0.006
SD of random effects: SD participant = 0.1, SD item = 0.002, SD residual = 0.3
Based on 2000 simulations, 80 Items, and 250 participants the estimated power was 0.815
Error rate: The parameters were set as follows:
Fixed effects: estimate intercept = -3.79, estimate probe word = -0.077, estimate validity = 0.153, estimate probe word * Validity = -0.13
SD of random effects: SD participant = 1.0, SD item = 0.8
Based on 2000 simulations, 80 Items, and 250 participants the estimated power was 0.949

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
Participants will be recruited via Clickworker. Screening criteria will be language (first language = German, Germany as the current place of residence), and age (between 18 and 35). Other screening criteria (not directly implemented in Clickworker but in our questionnaire) are reading-related disorders and right-handedness.
Transformation of data: Since reaction time data are often not normally distributed, the data will be log-transformed. If this transformation is not adequate, another or no transformation will be chosen.
Additional analysis: The 80 Items are presented in two separate blocks (40 items each). If the interaction of validity * probe word * block reaches significance, the two blocks will be analysed separately. Moreover, as exploratory analysis, we will compare reaction times/error rates for validity (valid vs. invalid) for the probe word right as well as the probe word wrong.
As another exploratory analysis, we will include the factor order of presentation (picture first vs. sentence first) in the linear mixed effects models to test whether the order of presentation influences the performances.