'Second pretest for text-picture validation study.'
(AsPredicted #65,477)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
The aim of this pre-study is to create a stimulus set consisting of sentences-picture pairs, where one sentence matches the picture well and one sentence does not match the picture well. The stimulus set used in this pre-study consists of 163 sentence-picture pairs. Each sentence-picture pair consists of one schematic drawing and either a sentence supposed to match (describing the picture correctly) or a sentence supposed to mismatch (describing the picture incorrectly) the picture.

3) Describe the key dependent variable(s) specifying how they will be measured.
Verification time per matching sentence-picture combination: Mean reaction time to judge a matching combination as correct; calculated per item and averaged over all participants.
Falsification time per mismatching sentence-picture combination: Mean reaction time to rate a mismatched combination as false (falsification time); calculated per item and averaged over all participants.
Wrong classification/error rate per item: Counting how often the items are assigned incorrectly (matching sentence-picture combinations rated as false; mismatched sentence-picture combinations rated as correct).

4) How many and which conditions will participants be assigned to?
Two randomized list of items: Every list includes one sentence-picture combination of every pair. This means every participant rates the validity of 163 sentence-picture combinations. In addition, 5 attention-check items are included. The attention-check items consist of a neutral picture (e.g. a schematic drawing of a bicycle) and a sentence, telling participants to rate this combination as false (respectively true). The participants will be randomly assigned to one of those two lists.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
Reaction time analysis
Verification time per matching sentence-picture combination: All items with a mean verification time below 1000ms or above 4000ms will be analysed further.
Falsification time per mismatched sentence-picture combination: All items with a mean falsification time below 1000ms or above 4500ms will be analysed further.
For further analysis, the reaction time distribution for the items meeting the criteria above will be analysed. It will be examined whether the mean reaction time for these items remain above or below the threshold when outliers are excluded. Outliers are defined as values above + 3 SD or below -3 SD.  

Error rate analysis
Based on the mean error rate of all participants, the average number of times an item is randomly assigned incorrectly is calculated. Items that are incorrectly assigned more often than expected based on this error rate will be analysed further. It will be checked whether only participants with low data quality misassigned the item, or whether several participants with high data quality also misassigned the item. Data quality is measured via the error rate of each participant and a self-assessment at the end of the questionnaire.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
Participants who do not agree to have their data processed will be deleted.
Subjects with an error rate of more than 25% in the trial run (before the experimental phase) must repeat the trial run. Participants who have to repeat the trial run more than two times will be excluded.
In addition, we will exclude participants who incorrectly answer at least three of our five attention check items.
Participants with an error-rate above 3 SD compared to the error rate of the other participants will be excluded.
Furthermore, trials with a reaction time above 10.000ms will be excluded. However, this will only be the case if 10.000ms is more than 3 SD higher than the average reaction time of the respective participant.
Finally, for the reaction time analyses only those trials with correct classification will be included.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
50 participants will be assessed (25 per condition).

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)