'SCC text messages templates: testing short vs. long templates' (AsPredicted #96,402)
Author(s) Alex Chohlas-Wood (Stanford University) - chohlasa@gmail.com Madison Coots (Stanford University) - mcoots@stanford.edu Emma Brunskill (Stanford University) - ebrun@cs.stanford.edu Sharad Goel (Harvard University) - sgoel@hks.harvard.edu
Pre-registered on 2022/05/08 20:05 (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? We are sending text message reminders to public defender clients to help these individuals plan for and remember upcoming court dates. These court dates are often mandatory, and failing to attend these court dates can result in eventual incarceration. For this experiment, we will test whether short, abbreviated reminders are more effective than the typical long-form reminders. Our hypothesis is that short reminders will be easier for clients to understand, resulting in higher response rates and appearance rates.
3) Describe the key dependent variable(s) specifying how they will be measured. We will examine whether shorter messages cause clients to respond at different rates, whether shorter messages cause clients to attend court at different rates, and whether shorter messages affect rates of eventual incarceration.
4) How many and which conditions will participants be assigned to? Participants will be assigned to one of two conditions: long-form reminders and short-form reminders.
Long-form reminders are the standard mode for reminding clients about their upcoming court dates. These templates often intend to include as much information as possible to help the clients prepare for the court date in an informed manner. For example, the following is an example of our 7-day long-form reminder: "Hi Alex. This is the Santa Clara County Public Defender. You have an upcoming court date on March 20, 2022 at 09:00AM. Please arrive fifteen minutes early at Department 1, HOJ - Main Courthouse located at 190 W Hedding St, San Jose, CA 95110 (Docket/Case No: C123456789,C223456789). If you have questions or concerns, call our office at (408) 299-7700. Reply with YES to confirm you will attend court."
Our short-form reminder template reduces the amount of included content, simplifies the presentation of this content, and more clearly prompts the client to respond (with double line breaks indicated with a forward slash): "This is the Santa Clara Public Defender. / Alex has a court date on Mon, Mar 20 at 09:00AM. / Please arrive 15 minutes early at Department 1, HOJ - Main Courthouse at 190 W Hedding St in San Jose. / Reply YES to confirm you will attend."
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. Logistic regression predicting one of the three outcomes listed above (response to reminder, appearance at court date, and eventual incarceration) as a function of treatment assignment (short vs. long texts) and other relevant covariates, e.g., demographic variables like age, case information like alleged charges, and known FTA history before the reminder. We will also aim to conduct a parallel set of analyses using models that can account for right-censored data, e.g., Cox models.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will exclude clients if our software backend officially assigns them to one condition but they erroneously receive reminders from another condition (we expect this will not happen).
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We will run this experiment for three months. At this point, we will conduct a preliminary analysis, and inform our agency partner if there is a need to continue running the experiment (e.g., if estimates on downstream incarceration are promising but noisy). If needed, we will consider extending the experiment to the end of the calendar year, at which point all clients will be assigned to the most effective treatment arm.
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will compare outcomes for the above two treatment arms against a control group which does not receive any reminders. This experiment started a couple months ago, and while we have not viewed the data yet, this comparison is not officially included in our pre-registration since the experiment has already started.
