'Information design in one-sided matching problems'
(AsPredicted #104269)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
(H1) Main hypothesis: Aggregate social welfare (=sum of payoffs of all group members) is the highest and equal to the first best under partial information, intermediate under full information, and the lowest under no information in the main scenario of interest, Scenario 1.

(H2) This is not true under an alternative scenario, Scenario 2, where the full information treatment results in the highest aggregate social welfare equal to the first best, and lowest under both no information and the partial information.

(H3) Given the properties of the respective scenarios, agents are predicted to follow the recommendation in 100% of the cases in Scenario 1, and follow the recommendation in Scenario 2 only when they are recommended the safe object, object A.

3) Describe the key dependent variable(s) specifying how they will be measured.
Two key outcome variables:

a) aggregate social welfare: sum of all subjects' payoffs
b) subject's choices: do subjects choose the safe or risky object (A or B), and, if applicable, do subjects follow the recommendation they received in the partial information treatment

4) How many and which conditions will participants be assigned to?
Three between-subject treatments:

a) No info: subjects will only receive information about the distribution of possible outcomes associated with risky choice B
b) Partial info: subjects will receive the same information as in No info, plus a "hint"/recommendation which object to choose. They will be informed that the hint is calculated by the computer that aims to maximize aggregate social welfare
c) Full info: subjects will know their exact payoff from choosing risky option B


Two within-subject treatments: all participants will complete two main tasks, i.e., make a choice under Scenario 1 and Scenario 2.
In Scenario 1 (ex-ante indifferent prior), the possible values for risky object B are 100, 300, or 500 points (with probabilities 20, 60, 20%).
In Scenario 2 (strong preference), the possible values for risky object B are 0, 200, or 400 points (with probabilities 20, 60, 20%).

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
Our hypotheses H1 and H2 consist of multiple inequalities, and are tested on the group level: we will primarily test these with the Jonckheere trend test, and secondarily pairwise with the Ranksum test with a MHT correction (see e.g., List et al., 2019, Experimental Economics).

Our hypothesis H3 considers individual behavior, and we will do a one-sample Mann-Whitney-U-test (vs. theoretical prediction of 100% hint-following in Scenario 1, 100% hint-following if recommended safe object A in Scenario 2, and 0% hint-following if recommended risky object B in Scenario 2).

Since we expect that not all subjects behave in accordance with our theoretical predictions, we will test whether their hint-following behavior changes from Scenario 1 to Scenario 2 with a within-sample Wilcoxon sign-rank test.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
We will not exclude outliers, but we will have rules for incomplete observations.

We expect that there will be subjects who will not complete the entire experiment. Since the experiment takes part in groups, we will adhere to the following exclusion & analysis rules:
- We exclude all groups where none of the group members completes the main part of the experiment (=Scenarios 1 & 2)
- If the number of subjects who fail to complete Scenarios 1 & 2 is relatively small, we will also exclude all groups where some of the subjects did not complete the main part of the experiment. We will only do this if our power does not drop under 65% by doing this.
- If the data loss is more severe if we were to exclude partially incomplete groups, the software will implement a random choice (50% chance of choosing A, 50% chance of choosing B) on behalf of the subjects whose choices are missing 24h after starting the experiment.

- We will not exclude any participants who complete the main part of the experiment (Scenarios 1 & 2) but fail to complete subsequent tasks; we will not impute their choices for those tasks.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
We ran simulations to show the required share of subjects "needed" for the partial information treatment to outperform full information treatment (H1), calculated the expected average effect size for three possible strategies the subjects could employ (always follow the recommendation, always do the opposite of the recommendation, always choose the safe option), and the required N to reach 80% power for the analysis done on group level. Similarly, we considered the average expected welfare gap between full info and no info, and calculated required N to reach 80% power to detect this difference on group level.

The resulting sample sizes are 257 groups in full info, 257 groups in partial info, and 41 groups in no info. Since we use 4-person groups, we aim to recruit 2220 subjects in total.

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
We are interested in the (behavioral) reasons why some groups do not reach the first best or why some subjects do not follow the recommendation even when it is in their interest to do so (or follow it when it is not in their interest to do so).

We collect data to distinguish between the following explanations:
a) Subjects do not pay attention: total time spent in the experiment (and time spent on instructions, and making choices), whether subject passes an attention check (if you are reading this, please select Australia)
b) Experimenter demand: we calculate the difference in behavior on the Bomb task when subjects complete the task without and subsequently with a direct request to behave in a particular way (please consider collecting as many boxes as you dare as this would be really helpful for our research)
c) Subjects do not understand instructions: we track how many comprehension questions subjects fail to answer and how many times
d) Subjects are unable to (Bayesian) update their beliefs based on the recommendation: we measure the extent to which subjects are able to calculate the probability of an event following an informative signal (cookie task)
e) Subjects are risk averse: we calculate subjects' risk aversion based on their first decision on the Bomb task (no explicit request to behave in a particular way)
f) Subjects are altruistic: we measure the subjects' altruism in a money sharing game (choose between bonus earnings for self or others)

Based on the subjects' behavior on the main and supplementary tasks, we classify which explanations are consistent with each subject's behavior.

As a robustness check, we will post hoc randomly reshuffle our subjects within groups such that different subjects' decisions become consequential for group outcomes, and verify that all our results are not a lucky outcome based on a specific subject ordering (again, using a MHT correction).