'Augmenting perceptual training with annotations and steps'
(AsPredicted #83157)
Created: 12/14/2021 07:22 PM (PT)
This is an anonymized version of the pre-registration. It was created by the author(s) to use during peer-review.
A non-anonymized version (containing author names) should be made available by the authors when the work it supports is made public.
1) Have any data been collected for this study already?No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Whether augmenting perceptual training with annotations and breaking the training down into steps is a more effective way to train people to do a difficult radiology task.
Hypothesis 1: Augmenting perceptual training with annotated feedback will be more effective than perceptual training without annotations.
Hypothesis 2: Modifying the perceptual training to break the task into steps will be more effective than standard perceptual training without steps.
We hypothesise that annotations will be more effective than steps, and thus predict the following ordering: annotations and steps will be the most effective condition, followed by the annotations only (no steps) condition, followed by the steps only (no annotations) condition, and then the standard perceptual training (no steps or annotations) condition.
3) Describe the key dependent variable(s) specifying how they will be measured. Each participant will complete a pre-training test and a post-training test. On each test trial, participants will grade an image according to a 7-point grading scale and we will calculate their error as the absolute distance from the correct answer. Averaging over the trials, we will calculate each participant's mean error for the pre-test and post-test, and then calculate the difference in error between the two tests.
4) How many and which conditions will participants be assigned to? Four conditions in a 2 (annotations vs no annotations) x 2 (steps vs no steps) between-participants design, with participants randomly assigned to conditions.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will conduct a 2 (annotations) x 2 (steps) between participants ANOVA, with the difference in the mean error between the pre-training and post-training tests as the dependent variable.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. At the start of the experiment, there will be a question designed to detect bots and any participants who fail this question will be excluded. Throughout the experiment, there will be four attention check trials (e.g., "Please respond with the '6' key") and participants who fail more than one of these will be excluded from the analysis. As we are interested in novice participants, we will also exclude any participants that indicate they have formal training or experience in radiology or with liver ultrasound images. Any participants that do not respond "yes" to having normal or corrected-to-normal vision will be excluded. We will exclude data from any repeat attempts and only analyse data from initial and complete attempts. Finally, we will also exclude data from participants if more than 5% of their data is either missing or corrupted (e.g., due to technical difficulties) or if participants reported other technical issues such as the images not loading properly throughout the experiment. In instances where less than 5% of a participant's data is missing, we will calculate performance from the remaining data.
7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined. We will aim to collect data from 50 participants per condition after exclusions (200 participants in total), although these numbers may vary slightly due to variance from the MTurk randomisation.
8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) Nothing else to pre-register.
