'Conversational AI & False Memories' (AsPredicted #162006)
Author(s) Pat Pataranutaporn (MIT Media Lab) - patpat@mit.edu Samantha Chan (MIT Media Lab) - swtchan@mit.edu Pattie Maes (MIT Media Lab) - pattie@media.mit.edu Aditya Suri (MIT Media Lab) - adity199@media.mit.edu Elizabeth Loftus (University of California, Irvine) - eloftus@uci.edu
Pre-registered on 02/13/2024 11:35 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Hypothesis 1: A suggestive conversational agent will induce a higher likelihood of false memories compared to a survey-based suggestive questionnaire.
Hypothesis 2: An improvisational conversational agent will induce a higher likelihood of false memories compared to a prescripted conversational agent.
Hypothesis 3: Factors such as age, gender, education level, and others moderate the effects of conversational agents on false memories.
3) Describe the key dependent variable(s) specifying how they will be measured. - Independent variables: method of inducing false memories (questioning)
- Dependent variables: False memory rate (% of false memories measured by a quiz)
- Moderating factors:
- Demographics: Age, education, Familiarity with technology and chatbots
- Attitude towards AI/Trust in AI
- Prior similar experiences: "Do you have experience related to the scenario?"
- Stress / Working memory - Cognitive Workload [Raw NASA TLX]
- Self-Reported interest in topic (crime investigations etc)
4) How many and which conditions will participants be assigned to? For each participant, one of the four conditions:
- No intervention
- Pre-scripted Conversational Agent
- Improvisational Conversational Agent
- Survey-based Questionnaire
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. The experimental protocol involved several steps. First, participants were shown a video to watch. Following the video, participants were randomly assigned to interact with one of four types of intervention conditions. After the interaction, participants answered a yes/no question related to the video content and answered the cognitive task load questions. Then, participants did a 5-minute distraction task (i.e., playing Tetris). Next, participants also responded to follow-up questions that measured memories and moderators. Finally, participants were asked to provide demographic information. For each of the outcome variables, we will analyze the results using descriptive statistics and Linear Regression/ANOVA models comparing the experimental conditions to a control condition.
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. We will prescreen participants fluent in English from the US ages 18 and above. We impose two check sections specifically for attention. Participants who fail such questions will be excluded as well as people who had a technical issue with the system and could not experience the AI system.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. 50 per condition = 200 participants
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) During recruitment, we will notify participants that they will be viewing CCTV footage depicting a criminal act. We will advise individuals who may feel distressed by such content that they are free to discontinue their participation. Participants will be informed of their ongoing right to voluntarily withdraw from the experiment at any time.
