'Opting-out and forced answering in sequential collaboration and independent judgments' (AsPredicted #113,490)
Author(s) Maren Mayer (Leibniz-Institut für Wissensmedien) - maren.mayer@iwm-tuebingen.de Daniel W. Heck (Philipps-Universität Marburg) - daniel.heck@uni-marburg.de
Pre-registered on 2022/11/18 00:25 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Replication of earlier findings for Sequential Collaboration:
Hypothesis 1: Participants in the Sequential Collaboration opt-out condition adjust presented judgments significantly less frequently than participants in the Sequential Collaboration forced condition.
Hypothesis 2: The magnitude of changes is larger in the Sequential Collaboration opt-out condition than in the Sequential Collaboration forced condition.
Hypothesis 3: The accuracy of individual judgments is (a) higher in the Sequential Collaboration opt-out than in the Sequential Collaboration forced condition, and (b) this difference becomes larger with increasing chain length.
Comparison of estimate accuracy:
Hypothesis 4: Final estimates obtained in the sequential collaboration opt-out condition are more accurate than final estimates obtained in the sequential collaboration forced condition. Moreover, estimates obtained with unweighted averaging of independent forced judgments are less accurate than judgments obtained with forced judgments in sequential collaboration.
3) Describe the key dependent variable(s) specifying how they will be measured. Change probability: For Hypothesis 1, the dependent variable is defined as the decision to adjust or maintain the presented judgment. Since participants in the forced condition must always provide a judgment, we consider changes in the presented position smaller than a Euclidian distance of 20 pixels or less as a proxy variable for a "maintained judgment".
Change magnitude: To test Hypothesis 2, we compute change magnitude as the distance between the presented judgment and the provided judgment. In the opt-out condition, we only include trials in this analysis in which the presented judgment was actually adjusted.
Judgment accuracy: For Hypothesis 3, we use the Euclidian distance between the provided judgment and the correct location as a measure of judgment accuracy. Similar to Mayer & Heck (2022), in the opt-out condition, the judgments of participants who decide to opt out of providing a judgment are replaced by the presented judgment.
Estimate accuracy: The final estimate obtained within a sequential chain is the latest available judgment for an item. To examine Hypothesis 4, we compute the Euclidean distance of this estimate to the correct location, and compare it to the unweighted average of individual judgments of independent forced judgments.
4) How many and which conditions will participants be assigned to? Types of questionnaires:
- Independent judgments forced: Participants are asked to place 57 cities on maps. There is no opportunity to skip items.
- Sequential collaboration forced: Participants are presented with judgments of previous participants for the 57 cities and must give an own judgment for each city.
- Sequential collaboration opt out: Participants are presented with judgments of previous participants for the 57 cities and can decide whether to adjust or maintain a presented judgment.
Conditions:
- Unweighted averaging forced: Participants are combined in groups of four and judgments are averaged to obtain location estimates for each city.
- Sequential collaboration forced: Sequences in this condition are formed with one participant who provided independent judgments (questionnaire "independent judgments forced") who serves as a starter for a sequential chain and three participants who answered the questionnaire "Sequential collaboration forced" and are required to provide new judgments consecutively.
- Sequential collaboration opt out: Sequences in this condition are formed with one participant who provided independent judgments (questionnaire "independent judgments forced") who serves as a starter for a sequential chain and three participants who answered the questionnaire "Sequential collaboration opt-out" and can (but do not have to) correct previous judgments consecutively.
No participant is assigned to more than one condition.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. All hypotheses will be examined using (generalized) linear mixed models with random intercepts for participants in Hypotheses 1, 2, and 3 (and for sequential chains / groups of participants in Hypothesis 4) and random intercepts for cities.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude participants that do not follow the instructions of the experiment by changing the browser window more than 5 times or by positioning more than five cities outside the highlighted countries. Moreover, we check whether participants position cities in the same area on all maps as a sign of careless responding. Lastly, we exclude judgments that are timed out after 30 seconds and the corresponding sequential chains in which they occur.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We performed a power analysis based on a previous study to determine the number of required participants. For an alpha of .05 and beta of at least 0.8 for each of the comparisons in Hypotheses 4, 800 participants are required. We, thus, recruit at least 800 participants.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) Nothing else to pre-register.
