'Benchmarking and accountability during the coronavirus pandemic' (AsPredicted #39240)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has 3 authors.
Pre-registered on 04/14/2020 08:30 PM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? This experiment studies whether and how citizen hold democratic governments accountable during the Covid-19 epidemic. There are two main sets of research questions:
(1) does exogenous variation in information about the response/performance of other countries (what we call benchmarking) affect individuals’ beliefs about how well their government has handled the coronavirus? Do benchmarking beliefs have a causal effect on willingness to reward/punish the incumbent government?
Theories of accountability suggest that in order to hold their governments accountable for how they respond to a crisis, voters can rely on (credible) information on how their country performed relative to other countries. Our hypothesis is that exogenous benchmarking information shapes’ people’s overall evaluation of the government. That is, providing a concrete favorable benchmark positively affects the global evaluation of how well the government has handled the crisis compared to an unfavorable benchmark. A corollary hypothesis is that benchmarking beliefs affect voting behavior.
(2) does an endogenous choice of a benchmark undermine accountability? Is there evidence of political biases in the choice of benchmarks, such that people more (less) inclined to support the government are more (less) likely to select a benchmark favorable to their views? Are people who select a particular benchmark unresponsive to countervailing information?
Theories of political behavior suggest that political pre-dispositions undermine accountability by, among others, affecting information acquisition and/or information processing. In the setting of our experiment with endogenous benchmarking, they predict that pre-treatment political preferences shape benchmark selection.
3) Describe the key dependent variable(s) specifying how they will be measured. (1a) Assessment of government performance in the crisis: Respondents are asked “Can you tell us how strongly you agree or disagree with the following statement? All in all, the government has handled coronavirus better than most other countries.” [Translation from country’s language.] Answers are recorded on a 11-point scale (0 = “strongly disagree”, 10 = “strongly agree”). Denoted by COMPGOV from now on.
(1b) Vote intentions: Measure 1 (placed several items after experiment in questionnaire) asks respondents how likely it is that their vote is influenced by how the government has handled the coronavirus crisis if an election were held in the near future (next week/Sunday). Responses on 11-point scale (0 = “Very unlikely”, 10 = “Very likely”). Measure 2 is a standard vote intention question, which records which party the respondent would vote for if an election were held next week/Sunday. The resulting measure will be equal to 1 if respondents are inclined to vote for the party or parties currently in government, 0 otherwise.
(2) Choice of the benchmark text in treatment condition 3. Binary variable equal to 1 if respondent selects more favorable headline, 0 otherwise.
4) How many and which conditions will participants be assigned to? Germany, UK, France:
Between-subject design. 3 treatment, 1 control condition.
Control group: receives no benchmarking information
Treatment group 1: receives exogenous benchmarking information indicating that their country’s government is doing better in response to the crisis than a benchmark country. Short vignette (no more than 100 words).
Treatment group 2: receives exogenous benchmarking information indicating that their country’s government is doing worse in response to the crisis than another country. Short vignette (no more than 100 words).
Treatment group 3: chooses benchmarking information by selecting one of two benchmarking headlines for further reading (positive or negative, as used for treatment groups 1 and 2).
In all treatment conditions respondents are asked to evaluate if text was (i) informative, (ii) credible, and (iii) if the they would share/recommend it.
Austria:
Between-subject design. 1 control condition,1 treatment condition (two stages)
Control group: receives no benchmarking information.
Treatment group: STAGE 1: respondents choose benchmark case by selecting one of two benchmarking headlines for further reading, a positive one (Austria as a leader in fight against coronavirus in Europe) or a negative one (Austria as a laggard). STAGE 2: Among those choosing the positive (negative) benchmarking headline, some receive (weak) counterbalancing information: Austria is a leader in fight against coronavirus in Europe but another country does similarly well (Austria is a laggard but another country in Europe has the same problem).
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. (1a) To test the basic benchmarking hypothesis, we regress COMPGOV on treatment indicators, using the negative benchmark as baseline. As stated above, the expectation is that positive benchmarking information leads to an increase in COMPGOV.
(1b) To test the corollary hypothesis regarding vote choice, we will use the fact that the experimental design generates an assignment instrumental variable. Two analyses: (i) An intention-to-treat analysis to estimate the effect of the exogenous benchmark on vote intention (using both measures as dependent variables). Implementation: regress vote intention on treatment indicator variables (with negative benchmark as baseline). (ii) The main quantity of interest is the causal effect of COMPGOV on vote intentions. Implementation: regress vote intentions on COMPGOV instrumented by treatment indicator variables. In IV analysis, we will report results with and without pre-treatment controls for socio-demographics (categories for age, gender, education, current employment status, family structure, region of residence, and current type of housing) as well as pre-treatment measures of news consumption (time spend on political news on an average weekday: none, less than an hour, 1-2 hours, 2-3 hours, more than 3 hours) and trust in media (dummy coding of 4-point scale).
(2) To test the hypothesis concerning the biased choice of benchmark information, we estimate a linear probability model with choice of the favorable benchmark as dependent variable. Beyond socio-demographics and the news consumption measure, an important explanatory variable is the pre-treatment satisfaction with how the executive (prime minister or president) has handled the coronavirus (measured on an 11-point scale).
In the case of Austria, the same analysis of benchmark choice will be conducted. However, given the difference in experimental design the effect of exogenous benchmarking information on the evaluation of government performance will be estimated conditional on choosing a generally positive/negative benchmark.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. No cases will be classified or excluded as "outliers".
In every analysis cases with item non-response will be excluded and reported.
By design, the analysis of endogenous benchmarking can only be conducted for treatment group 3 (treatment group 1 in Austria).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. The experiment is embedded in an opt-in online panel for the cooperative survey project on Citizens’ attitudes to Covid-19 run by the international survey company Ipsos. Ipsos will attempt to balance the panel sample to be representative of each country’s population of eligible voters.
Target sample sizes:
N=2,000: Germany, France
N=1,000: UK, Austria
Sample size differences are due to resource constraints unrelated to the experiment
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) Secondary analyses:
Treatment effect heterogeneity: Does effect of exogenous benchmark treatments on global evaluations vary by trust in media (4-point scale), political news consumption, pre-treatment satisfaction with the prime minister, and pre-treatment satisfaction with how democracy is working in the country?
Related to theories of political behavior, we will assess if respondents exposed to positive (negative) exogenous benchmarking information will be more (less) inclined to evaluate the text positively (informative/credible/willing to share) if they are pre-disposed toward (against) the government.
