As Predicted:BCFG Flu Shot Mega-Study (#47510)
Created: 09/11/2020 08:40 AM (PT)
This is an anonymized version of the pre-registration. It was created by the author(s) to use during peer-review.
A non-anonymized version (containing author names) should be made availble by the authors when the work it supports is made public.
1) Have any data been collected for this study already?
No, no data have been collected for this study yet.2) What's the main question being asked or hypothesis being tested in this study?
We plan to run a field experiment to test and compare the effects of 19 different interventions aimed at encouraging people to take the flu shot relative to a placebo control condition.3) Describe the key dependent variable(s) specifying how they will be measured.
Our field experiment will be conducted with Penn Medicine and Geisinger Health patients via SMS messages sent prior to their first primary care appointment during the study period (see Section 7), hereafter referred to as the “target appointment.” The key dependent variable is whether participants receive a flu shot at or before their target primary care appointment (as recorded in their electronic health records). The secondary dependent variable is whether they receive the flu shot at or before their target primary care appointment or at any time after this appointment (by March 31, 2021), as recorded in their electronic health records. For both dependent variables, flu shots received more than three days before the participant’s target primary care appointment took place will be excluded.
If participants cancel or do not show up for their target appointment after they have been randomized to a treatment and then schedule a new appointment during the study period (see Section 7), their new primary care appointment becomes the target appointment.
Participants who have been randomized to a treatment will be counted as not having received a flu shot at their target appointment if they cancel or do not show up for their appointment and do not schedule a new appointment during the study period.4) How many and which conditions will participants be assigned to?
Participants will be randomly assigned to one of 19 different experimental conditions or a holdout control condition, and random assignment will be stratified by site (Penn Medicine versus Geisinger Health), age group (age 18-64 versus age 65+), and whether the participant received a flu shot last year according to their medical records (yes versus no/unknown). The study conditions are:
1. Default Reservations Opt-Out Condition
2. Default Reservations Opt-In Condition
3. Intergroup Competition Treatment Condition
4. Intergroup Competition Control Condition
5. Flu Shot for You Symbolic Condition
6. Flu Shot for You Herd Immunity Condition
7. Flu Shot for You Control Condition
8. Prosocial Condition
9. Self-Oriented Condition
10. Information Vivid Condition
11. Information Basic Condition
12. Information Control Condition
13. Sharing Humor Condition
14. No Humor Condition
15. Healthy Habits Easy Health Behavior Condition
16. Healthy Habits Difficult Health Behavior Condition
17. Healthy Habits Control Condition
18. Just-In-Time Reminders 24-Hour Condition
19. Just-In-Time Reminders 15-Minute Condition
20. Holdout Control Condition
Descriptions of the study conditions are available here: https://www.dropbox.com/s/ehiggy72k0qk4l9/BCFG%20Flu%20Shot%20Mega-Study%20Conditions_PennGeisinger.pdf?dl=05) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
We will evaluate the effects of the interventions as follows:
For the main analysis, we will run an ordinary least squares regression (OLS) to predict whether a given participant received a flu shot at or before their target appointment (a binary indicator variable). The primary predictor variables in our regression will be 19 indicators for assignment to each of the study’s 19 experimental conditions (an indicator for the holdout control condition will be omitted). Our regression will include the following control variables:
- Indicator for being a Penn Medicine patient
- Participant age at target appointment
Indicators for participant race/ethnicity (Black non-Hispanic, Hispanic, Asian, white non-Hispanic, other/unknown; white non-Hispanic omitted) and gender (male, female, other/unknown; female omitted);
- Indicator for having received a flu shot in the 2019-20 flu season according to participant’s medical records
- Indicators for the type of provider who saw the patient (attending/faculty physician, resident, nurse practitioner or physician assistant; nurse practitioner omitted)
- Days separating participant’s target primary care appointment and start of study
- Days separating participant’s target primary care appointment and start of study squared
We will also conduct this analysis (1) without any control variables, (2) excluding patients with target appointments with residents, nurse practitioners or physician assistants, (3) separately for patients with target appointments between September and December versus between January and March (if recruitment continues past December; see section 7 for details), and (4) including the following additional control variables:
- Indicator for whether participant is a. married, b. has children
- Indicators for participant insurance type (Medicaid, Medicare, commercial/other, none/unknown; commercial/other omitted)
- Indicators for diagnoses of a. diabetes, b. asthma, c. high blood pressure, d. high cholesterol
- Patient Charlson Comorbidity Score
- Participant body mass index
- Indicator for whether the patient is a known smoker
- The patient’s number of medical appointments in the previous year
- An indicator for having received a flu shot in the a. 2015-16 flu season, b. 2016-17 flu season, c. 2017-18 flu season, d. 2018-19 flu season
- Fixed effects for week of the year when the target appointment takes place
- Median household income in the participant’s zip code
- Indicator for whether the target appointment was the patient’s first appointment
- Indicator for whether the participant has previously tested positive for influenza
As a robustness check, we will re-run all of the above analyses using logit regressions instead of OLS regressions.6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
No outliers will be excluded. If there are severe technological malfunctions (e.g., a participant receives no messages, the wrong messages, or repeat messages), these observations will be excluded.7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
We will recruit as many patients as possible with new or routine (non-sick) primary care appointments at Penn Medicine and Geisinger Health starting in September 2020. Participants will be enrolled in one of several studies launching in parallel if they meet the following criteria:
1. They have a cell phone number recorded in the Penn Medicine database or the Geisinger Health database
2. They have not opted out of receiving SMS appointment reminders
3. They do not have a documented allergy or adverse reaction to the flu vaccine
4. They have not already had a flu vaccination in 2020, as documented in their electronic health records.
We will stop enrolling participants with appointments scheduled to occur after December 31, 2020 if we have reached 4,000 participants per condition. If we do not have 4,000 participants per condition by December 31, 2020, we will continue enrolling participants who have appointments scheduled to occur up until March 31, 2021 (inclusive). A sample size of 4,000 participants per condition will give us 80% power to detect a 3% difference in flu shot uptake between conditions.8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
We will also conduct heterogeneity analyses where we repeat the primary analyses on different participant subgroups for each of the control variables listed in the main analysis (see Question 5). Specifically, we will run subgroup analyses by age, gender, race/ethnicity, type of healthcare provider, whether the participant is a Penn medicine patient, whether the participant received a flu shot in the 2019-20 flu season, and number of days separating the participant’s target primary care appointment from the start of the study.