'FJC/Harvard 2017 Experiment on Judicial Decision-Making' (AsPredicted #3,688)
Author(s) Holger Spamann (Harvard Law School) - hspamann@law.harvard.edu Daniel Klerman (USC Gould School of Law) - dklerman@law.usc.edu
Pre-registered on 2017/04/10 05:20 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet
2) What's the main question being asked or hypothesis being tested in this study? Judges' choice of law decision will be influenced by (1) the relevant location parameters identified by the applicable conflicts law, and (2) by its impact on the more sympathetic party. These effects will be mitigated by the type of conflicts law: (1) will be (much) stronger under a rule than a standard, to the point of eliminating (2) altogether.
3) Describe the key dependent variable(s) specifying how they will be measured. The key dependent variable is the law chosen by the judge. However, the way to code the law differs by effect:
1) For measuring the impact of location parameters, the coding is 1 if the judge chose the law of the state of the accident and the forum is Wyoming, or if the judge chose the law of the state of common domicile and the forum is South Dakota
2) For measuring the impact of sympathy, the coding is 1 if the judge chose the law of Nebraska and the defendant (Rogers) is sympathetic, or if the judge chose the law of Kansas and the defendant is not sympathetic.
4) How many and which conditions will participants be assigned to? 8 conditions: 2x2x2 factorial.
Factor 1: Forum: Wyoming or South Dakota. Wyoming follows lex loci delicti (law of place of accident applies). South Dakota follows 2nd restatement (which points towards the law of common domicile).
Factor 2: Accident state: Kansas or Nebraska. The parties reside in the respective other state.
Factor 3: Sympathies: Either the defendant or the plaintiff is sympathetic, the other is not.
We block the randomization on judge type (circuit, district, bankruptcy, or magistrate), and randomize only 2 judges from each type into the Wyoming level of factor 1 (because we expect the residual variance under that factor to be zero or close to zero).
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. 1) Main effects of factors 2 and 3, separately for both levels of factor 1. Based on our priors and at alpha=10%, we have 80% power or more for the law main effect under the standard, and virtually 100% under the rule. We do NOT expect to find a sympathy main effect under the rule, and have only 50% power or less under the standard.
2) Differences in effect sizes for factors 2 and 3 between both levels of factor 1. Given that we expect the residual variance under the rule to be zero (because the law fully determines the outcome), we have 80% or more power to identify the difference in law main effects between the rule and the standard condition. We do not have power to test formally a difference in sympathy main effects between the rule and the standard, but we will report the test nevertheless.
We will use randomization based Fisher tests, and report p-values and "fiducial intervals" (see Imbens and Rubin 2014).
6) Any secondary analyses? Variance: we will test for differences in residual variance under the standard and the rule -- it should be lower under the rule.
Bayesian posteriors: We will calculate them as in a working paper version of Spamann and Kloehn (2016). We will report them depending on audience reception.
Robustness: Our main tests use all observations, but we will also show results without participants who indicated that they had participated in, or heard about, a similar study that one of us conducted in the past (Spamann and Kloehn, Journal of Legal Studies 2016), and/or who indicated that they did not think the case seemed realistic.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. All judges who agree to participate at a workshop at Harvard Law School on April 10, 2017. If less than 40, we will try to find more at another workshop.
8) Anything else you would like to pre-register? (e.g., data exclusions, variables collected for exploratory purposes, unusual analyses planned?) We also ask participants (1) what percentage of other participants they think decided the case they did, and (2) what they think the study was about.
This is a modification of a prior aspredicted.org submission (#3686), which we abandoned because we realized after approval by one of us (Spamann) that we had described the wrong analysis procedure.
