'The role of opting out vs. forced answering in Sequential Collaboration'
(AsPredicted #105,467)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
Hypothesis 1: Participants in the opt-out condition adjust presented judgments significantly less frequently than participants in the forced condition. Since previous studies showed that even though opting out of providing a judgment is not always used, we first test whether participants in the opt-out condition use this possibility.
Hypothesis 2: The magnitude of changes is larger in the opt-out condition than in the forced condition.
Hypothesis 3: Judgment accuracy is (a) higher in the opt-out than in the forced condition, and (b) this difference becomes larger with increasing chain length.
Hypothesis 4: Final estimates obtained in the opt-out condition are more accurate than final estimates obtained in the forced condition. The test of this hypothesis is not statistically independent from an examination of Hypothesis 3. However, we added this hypothesis as a critical element of the research question on the role of opting out in sequential collaboration.

3) Describe the key dependent variable(s) specifying how they will be measured.
Change probability: For Hypothesis 1, we consider each participant's decision to adjust or maintain the presented judgment as the dependent variable. Since participants in the forced condition have to provide a judgment, we consider changes in the presented position smaller than a Euclidian distance of 20 pixels or less as a proxy for a "maintained judgment".
Change magnitude: To test Hypothesis 2, we compute change magnitude as the distance between the presented judgment and the provided judgment. In the opt-out condition, we only include trials in this analysis in which the presented judgment was actually adjusted.
Judgment accuracy: For Hypothesis 3, we use the Euclidian distance between the provided judgment and the correct location as a measure of judgment accuracy. Similar to Mayer & Heck (2022), in the opt-out condition, the judgments of participants who decide to opt out of providing a judgment are replaced by the presented judgment.
Estimate accuracy: The final estimate obtained within a sequential chain is the latest available judgment for an item. To examine Hypothesis 4, we compute the Euclidean distance of this estimate to the correct location.

4) How many and which conditions will participants be assigned to?
In both conditions, participants form sequential chains of three participants in which the first participant encounters fixed, preselected location judgments of 57 European cities of varying accuracy. The following contributors encounter location judgment updated with the judgments provided by the previous participant.
- Opt-out condition: Typical sequential collaboration paradigm; participants encounter a judgment of a previous participant (or preselected judgments) and can decide whether to adjust or maintain the presented judgment.
- Forced condition: Sequential collaboration, but participants must provide a judgment for each item and cannot opt-out of providing a judgment.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
All hypotheses will be examined using (generalized) linear mixed models with random intercepts for participants and items.
To test Hypothesis 3, we additionally set mean-centered contrasts for position in the sequential chain to test for the decrease in distance to the correct location. To account for the nuisance variance due to the accuracy of the (preselected) location judgments presented to the first participant in a sequential chain, we include the distance of those judgments to the correct location as a covariate in the analysis.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
Already during participation, we exclude participants who change the browser window more than five times.
After data collection, we check the resulting location judgments for a) irregular answer patterns (e.g., many judgments in one specific map region such as corners) and b) many extremely accurate judgments since such answer patterns may indicate looking up answers during participation. Participants with such answer patterns are excluded as are those sequential chains they participated in. Previous studies showed that such behavior is, however, rare.
We additionally check whether judgments are timed out after 30 seconds and exclude these judgments and sequential chains built with this judgment.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
Based on the linear mixed model used for testing Hypothesis 3 and using data obtained in a previous study, we conducted a power analysis by simulating larger sample sizes. This analysis indicates a power of .85 for 35 sequential chains per condition, .89 for 40 chains per condition, and .92 for 45 chains per condition. Thus, we aim for at least 40 sequential chains in each condition resulting in 240 participants (2 conditions * 40 chains * 3 participants per chain).

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
Nothing else to pre-register.