'Text-picture activation-validation study with successive order and longer texts'
(AsPredicted #129,169)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
This study is closely related to the study "Online sentence-picture reactivation-validation experiment with successive order" (#101349). We investigate whether text-picture combinations are reactivated and validated automatically. In contrast to study #101349, we will now use longer text segments instead of single sentences.
Each text-picture combination can be valid (a short story consisting of four sentences ending with a sentence matching the picture) or invalid (a short story consisting of four sentences ending with a sentence not matching the picture). The combinations are presented successively: First, participants see the short text or the picture. Second, they have to respond to the probe word "wrong" by pressing the key D or to the probe word "right" by pressing the key K. Third, participants have to solve a simple arithmetic problem. Fourth, the picture or short texts that corresponds to the short text or picture presented before is presented. After that, participants have to react again to the probe words "wrong" or "right". Please note that the hypotheses described below refer to the reaction time and error rate regarding this last probe.
We hypothesize that the reaction time after congruent stimuli (text-picture combination is valid, probe word is right or text-picture combination is invalid, probe word is wrong) is faster than after incongruent stimuli (text-picture combination is invalid, probe word is right or text-picture combination is valid, probe word is wrong). This should result in a significant interaction between validity and probe word meaning that for valid text-picture combinations faster reaction times regarding the reaction to the probe word "right" than to the probe word "wrong" are expected, whereas for invalid text-picture combinations faster reaction times for the probe word "wrong" than for the probe word "right" are expected. Looking at the interaction the other way around this means that for the probe word "right" the reaction time is slower after invalid items than after valid items, whereas for the probe word "wrong" the reaction time is slower after valid than after invalid items.
Moreover, we hypothesize that the error rate after congruent stimuli is lower than after incongruent stimuli. This should result in a significant interaction between validity and probe word, meaning that for valid text-picture combinations less errors regarding the reaction to the probe word "right" than to the probe word "wrong" are expected, whereas for invalid text-picture combinations less errors for the probe word "wrong" than for the probe word "right" are expected. Looking at the interaction the other way around this means that for the probe word "right" the error rates are higher after invalid items than after valid items, whereas for the probe word "wrong" the error rates are higher after valid than after invalid items.

3) Describe the key dependent variable(s) specifying how they will be measured.
Reaction time: Time until a participant presses the key D or K after seeing the probe word measured in milliseconds. Please note that in the main analysis described above only the reaction time for the second probe word is included.
Error rate: How many times a participant presses the key D after seeing the probe word right and presses the key K after seeing the probe word wrong. Please note that in the main analysis described above only the error rate for the second probe word is included.

4) How many and which conditions will participants be assigned to?
There are four within-subjects conditions resulting from crossing the factor text-picture validity (valid vs. invalid) and the factor probe word (right vs. wrong).
We will use eight lists of items that counterbalance for the factors validity, probe word, and order of presentation (i.e., which part of the item is presented first, text or picture): Participants are randomly assigned to one of those eight lists. Every list contains 64 text-picture combinations. The number of valid and invalid text-picture combinations, the number of the probe words, and the number of congruent and incongruent trials are the same in all lists. Moreover, after 16 randomly selected trials, participants have to solve an attention check task and either indicate whether they have just seen this picture or answer a short yes-no comprehension question about the text. The list will be presented in two blocks à 32 trials each.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
Reaction time: Linear mixed effects model with item and participant as random effects, probe word, and validity as fixed effects will be conducted. If the interaction of validity * probe word is significant, pairwise single comparisons will be conducted. We will compare a) reaction times for probe words (wrong vs. right) for invalid as well as valid stimuli, and b) reaction times for validity (valid vs. invalid) for the probe words right vs. wrong.
Error rate: General linear mixed effects model with item and participant as random effects, probe word, and validity as fixed effects. If the interaction of validity * probe word is significant, pairwise single comparisons will be conducted. We will compare a) error rates for probe words (wrong vs. right) for invalid as well as valid stimuli, and b) error rates for validity (valid vs. invalid) for the probe words right vs. wrong
In case the interactions are not significant, we will run additional analyses to test whether the single comparisons reach significance.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
Exclusions of participants: Participants who do not agree to have their data processed will be deleted. Participants have to speak German fluently and be right-handed. Participants who report serious technical issues during the study will be excluded. Moreover, participants with more than 40% incorrect responses to the probe words (right and wrong) will be excluded. Participants with an error rate above 40% in the attention check tasks will be excluded.
We will exclude reactions towards the second probe word if the reaction time was under 10ms or above 5s. For reaction time analysis, only correct responses will be included.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
We conducted a power simulation based on the data of previous studies (#101349, and #89392). Validity and probe word are contrast coded (invalid = 1, valid = -1; wrong = 1, right = -1) and reaction time data is log transformed.
The simulations revealed that 250 participants lead to sufficient power. Hence, we plan to recruit 300 participants. If, however, the drop-out based on our beforementioned criteria is higher than expected, we will recruit participants until we have approximately 250 valid cases.
The parameters for the reaction time analysis were set as follows:
Fixed effects: estimate intercept = 6.58 estimate probe word = 0.0067, estimate validity = 0.0095, estimate probe word * Validity = -0.007
SD of random effects: SD participant = 0.24, SD item = 0.023, SD residual = 0.3
Based on 2000 simulations, 64 Items, and 250 participants the estimated power was 0.85
Error rate: The parameters were set as follows:
Fixed effects: estimate intercept = -4.39, estimate probe word = -0.07, estimate validity = 0.2, estimate probe word * Validity = -0.2
SD of random effects: SD participant = 1.5, SD item = 1.0
Based on 1000 simulations, 64 Items, and 250 participants the estimated power was 0.989

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
Participants will be recruited via Clickworker. Screening criteria will be language (first language = German, Germany as the current place of residence), and age (between 18 and 35). Other screening criteria (not directly implemented in Clickworker but in our questionnaire) are reading-related disorders and right-handedness.
Transformation of data: Since reaction time data are often not normally distributed, the data will be log-transformed. If this transformation is not adequate, another or no transformation will be chosen.
Additional analysis: The 64 Items are presented in two separate blocks (32 items each). If the interaction of validity * probe word * block reaches significance, the two blocks will be analysed separately.
In addition, we want to explore whether the difference in reaction times and error rates are due to the validation process. As the items are only valid or invalid after both item parts (i.e., text and picture) have been presented, the reaction towards the first probe word can serve as a baseline. After applying the same data cleaning steps for the reaction towards the first probe word as for the second probe word (i.e., probe words with reaction times under 10ms or above 5s are excluded; for reaction time analysis, only correct responses will be included), we will include the factor position probe word (first probe word vs. second probe word) as dichotomous factor in the (general) linear mixed effect models. We expect a significant interaction between the factors validity, probe word and position probe word. We will further analyse separately for each position of probe word (first vs. second) whether the interactions between the factors probe word and validity are significant.
As another exploratory analysis, we will include the factor order of presentation (picture first vs. text first) in the linear mixed effects models to test whether the order of presentation influences the performances.