Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has 3 authors.
Pre-registered on 01/13/2023 08:37 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? 1) Does contingent belief updating (updating beliefs for all possible signal realizations) reduce or increase the bias in belief updating?
2) How does the effect of contingent belief updating depend on the characteristics of the signal-generating process?
3) Describe the key dependent variable(s) specifying how they will be measured. 1) Bias: The absolute distance of the elicited posterior and the Bayesian posterior
2) Underinference: The coefficient from regressing the elicited log posterior odds (ratio of posteriors for both states given the same signal) on the correct log posterior odds given a signal realization.
4) How many and which conditions will participants be assigned to? The main task of the experiment is based on a "balls-and-urns" updating exercise, as follows: Subjects are asked to consider two bags, A and B, both equally likely to be selected. In each bag, there are blue and orange balls. Subjects know the distributions of balls in both bags. First, subjects click on "select bag", and a bag is selected by a fair coin flip; then, subjects click on "draw ball", and a ball is drawn randomly from the selected bag. The subjects' task is to guess the probability of each bag being selected. Subjects do this task for 10 rounds; in each round, the signal-generating process is different. Depending on the treatment, we vary the belief elicitation. There are three between-subject treatment conditions.
1. Treatment Direct (Conditional Belief Updating)
In treatment Direct, beliefs are elicited conditional on the realized signal, when the subject already has observed the color of the drawn ball. Therefore, only one signal – the realized one – is considered by the subject. This corresponds to the classic "balls-and-urns" tasks.
2. Treatment Strategy (Contingent Belief Updating)
In Strategy, beliefs are elicited for both possible signal realizations. As in Hypothetical Direct (below), subjects have not observed the realized signal, but subjects are asked to consider both cases: (1) the computer draws an orange ball, and (2) the computer draws a blue ball.
3. Treatment Hypothetical Direct
In Hypothetical Direct, beliefs are elicited conditional on only one hypothetical signal when subjects have not observed the signal realization yet. Subjects are asked to consider one of the following hypothetical cases: (1) the computer draws an orange ball, or (2) the computer draws a blue ball. Each case is chosen with equal probability and it is randomly chosen for each round.
The comparison between Direct and Strategy is our main focus to study the effect of contingent thinking on belief updating. The third treatment allows us to decompose the effect of contingent thinking into hypothetical thinking and contrast reasoning.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. We will use regressions to test our hypotheses, clustering standard errors on the individual level. We will include controls for different signal-generating processes.
To test how the bias depends on the treatment, the bias will be regressed on a set of treatment indicators. To test the effect on underinference, we regress the elicited posterior odds on the correct log posterior odds, interacting this with treatment indicators to test whether underinference is affected by treatment.
We will test whether there are heterogenous treatment effects depending on the characteristics of the signal-generating process by interacting the treatment indicators separately with indicators of whether the signal is symmetric, and a measure of the signal strength.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. After subjects read the experimental instructions, they answer two sets of control questions. We define that a subject fails a set of control questions if there is at least one mistake. We exclude subjects that fail at least one of the sets of control questions three times.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. 150 observations per between-subject treatment, so 450 in total.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will explore how the effect of contingent belief updating depends on subject characteristics, such as their degree of cognitive reflection, and on a measure of cognitive uncertainty. Furthermore, we will study how contingent belief updating affects the consistency of belief updating.
In subsequent experiments, we will explore the role of contingent belief updating in other domains.
