'Achievement Goals and Progress-Feedback Systems in an E-Learning Tool'
(AsPredicted #95047)
Download PDF

---

---

1) Have any data been collected for this study already?
No, no data have been collected for this study yet.

2) What's the main question being asked or hypothesis being tested in this study?
RG 1: We intend to investigate the association between achievement goals with behavior after progress feedback (PF), expecting:
H1a: Closeness of attainment- and loss of a medal, as indicated by the PF, will be associated with learning behaviour, so that low closeness to medal attainment/loss will predict fewer session stops/switches after PF.
H1b: Both stronger task approach (TAP) and stronger performance approach (PAP) goals will predict fewer session stops/switches after PF indicating nearing attainment of a medal.
H1c: Both stronger task avoidance (TAV) and stronger performance avoidance (PAV) goals will predict fewer session stops/switches after PF indicating nearing loss of a medal.
H1d: With regards to H1b, we predict an interaction effect of TAP and PAP goals with the closeness to attainment indicated by the PF, where stronger goals predict fewer session stops/switches after PF more, the closer the attainment of a medal is as indicated.
H1e: With regards to H1c, we predict an interaction effect of TAV and PAV goals with the closeness to loss indicated by the PF, where stronger goals predict fewer session stops/switches after PF more, the closer the loss of a medal is as indicated.

RG 2: We intend to investigate the association between achievement goals with attitudes shown towards the PF and the e-learning itself, expecting:
H2a: Both higher TAP and higher PAP goals will predict more positive perceptions of the PF.
H2b: Both higher TAP and higher PAP goals will predict a more positive usability evaluation of the e-learning system.
H2c: Both higher TAP and higher PAP goals will predict higher continuance intention of the e-learning system.
H2d: With regards to H2c, we predict an interaction effect of TAP and PAP goals with the number of medals attained, where higher goals predict continuance intention more, the higher the number of medals attained by the individual.

RG 3: We intend to investigate the association between achievement goals with behaviour shown with regards to receiving medals as part of the progress system, expecting:
H3a: Both stronger PAP and TAP goals will predict fewer session stops/switches after receiving a medal.
H3b: Both stronger PAP and TAP goals will predict higher learning time after receiving a medal.

For all research questions we will additionally explore associations of the achievement goals with e-learning behaviour not explicitly stated in the hypotheses.

Please find additional research goals in section 8 (Other).

3) Describe the key dependent variable(s) specifying how they will be measured.
Learning behavior: We will use the log files provided by the e-learning software to conduct the respective parameters as follows:
Learning time after receiving a medal will be measured by learning ime in the system after receiving a medal until stopping or switching the session. Note that users will be automatically logged out after 10 minutes of inactivity. This 10 minutes are not included in total learning time.

Session stops and switches indicate whether learners stopped/switched between PF.

Attitudes and perceptions of the PF and the e-learning system will be measured with the following self-report measures in a follow-up questionnaire.
Perceptions of the PF system will be measured with an adaptation of the badge impact survey (Biles et al., 2014; Kyewski & Krämer, 2018).
Continuance intention of the e-learning tool will be measured with an adaptation of a 3-item continuance intention survey for e-learning systems (Roca et al., 2006).
Usability evaluation of the e-learning system will be measured with an adaptation of the (10-item) System Usability Scale (SUS; Brooke, 1996).

4) How many and which conditions will participants be assigned to?
There is only one general condition and we will not conduct between subject manipulations. However, one of two different types of PF will be presented after every fifth exercise. If the progress is over 50% towards the next medal, an attainment frame will be selected, if the progress is under 50% a loss frame will be selected. No feedback will be provided, if progress is under 50% and no medal has been achieved and if progress is over 50% but all medals are already achieved.

5) Specify exactly which analyses you will conduct to examine the main question/hypothesis.
We will run (hierarchical) regression analyses (moderated) mediation models to evaluate our research questions to evaluate our research questions.

6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations.
We will check the data for outliers (exceptionally high amount of learning time and exceptionally low amount of learning time) as well as behavioral patterns that indicate that the software was not used properly (e.g., some learning activity after purchase followed by little activity over time). To test the robustness of our effects, we will explore whether excluding extreme cases or cases indicating improper use of the software will affect the results.

We will also conduct further exploratory inspections into the data searching for further parameters that might help us to understand both learning patterns and software use better. These inspections can guide further data exclusion when investigating our main hypotheses.

7) How many observations will be collected or what will determine sample size?
No need to justify decision, but be precise about exactly how the number will be determined.
The sample size is limited by the number of users that purchase the software in time span between 25.04.2022 and 10.06.2022. We will include all users that purchase the software and who provide informed consent in our sample.

8) Anything else you would like to pre-register?
(e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?)
RG 4:We assume that the association between all measured achievement goals and e-learning behavior is moderated by the level of the current (medal-)level, learning time and experience with the progress system, so that the association between achievement goals and learning behaviour is weaker, the higher those parameter are (H4).

RG 5: We expect students' beliefs about the utility value of engagement in e-learning for the pursuit of the respective achievement goal to bolster positive associations between achievement goals and learning behaviour, as well as diminish negative ones (H5).

Demographic variables (age, gender) will be assessed before usage of the e-learning tool.
Previous experience with the e-learning tool will be derived from system data.

We will assess participants' intrinsic motivation towards the study subject they are using the e-learning system for with a 6-item scale, to include as a control measure.

We will assess participants' achievement goals using the respective subscales of the questionnaire by Daumiller (2019). We will only use 3 instead of 4 items per subscale. Based on these original items, we also developed items measuring participants' beliefs about the utility value of the e-learning software for the pursuit of all eight measured achievement goals (3 items per subscale).

Parallel to this study, another research project is evaluating how achievement goals and the beliefs about the utility of the software regarding these goals affect parameters describing different learning behavior in e-learning systems and how these parameters predict exam performance. Therefore, in the joint questionnaire of both research projects participants are asked for their student identification number. This will be used by the other research project to match exam performance afterwards.