'A cross-cultural examination of an SJT between Cuba and Germany' (AsPredicted #95372)
Author(s) This pre-registration is currently anonymous to enable blind peer-review. It has 4 authors.
Pre-registered on 04/27/2022 08:36 AM (PT)
1) Have any data been collected for this study already? No, no data have been collected for this study yet.
2) What's the main question being asked or hypothesis being tested in this study? Most Situational Judgment Tests (SJTs) are developed in English-speaking contexts; changing the cultural context might also change how the SJT works. Personal initiative is an important skill for performing in a 21st century workplace (Frese & Fay, 2001). Hence, in our study, we investigate the performance of an SJT on personal initiative (Bledow & Frese, 2009; 12 items, developed in Germany) in Cuba. As German and Cuban work contexts differ, it is promising to investigate whether this SJT can be used to measure personal initiative in Cuba in a valid way. For that purpose, we will compare the SJT's psychometric properties (reliability, factor structure, item properties, convergent validity, measurement invariance). Thus, we aim to investigate participants' perceptions of this SJT and personnel selection practices in Cuba.
RQ1: How are the psychometric properties of a SJT on personal initiative in Cuba, compared to a German context?
The SJT on personal initiative was developed and validated in a German context (Bledow & Frese, 2009). Hence, this SJT might be more representative of situations in German workplaces as in Cuban workplaces and, thus, the typicality of the given situations might be perceived lower in a Cuban sample. Further, we are aware of no reports concerning the use of SJTs in Cuban organizations and enterprises: Cuban participants might not be as familiar with SJTs as a German sample, which might reduce the validity perceptions of this measure. Hence, we suggest:
H1: Applicants' perceptions of situation typicality, procedural fairness dimensions (face validity, perceived predictive validity, opportunity to perform, and perceived knowledge of results), positive affect (enjoyment), and test-taking motivation will be higher for German test-takers than Cuban test-takers.
Krumm et al. (2015) removed situation descriptions from SJT-items: Surprisingly, for a majority of the items it did not make a difference whether the situation was included or not. As German participants might be more familiar with the situations described in this SJT, they might be able to infer the situations without situation descriptions. However, Cuban participants might not be as familiar with these situations as their German counterparts and, thus, could come to different conclusions about what kind of situation is intended by this item. Hence, Cuban participants might rely more on situation descriptions than German participants. Therefore, in this study, we aim to replicate this finding in the Cuban context to gain insights into the universality of Krumm and colleagues' findings. We suppose:
H2: Situation Descriptions are more important for Cuban participants to identify the correct answer in a SJT on personal initiative than in a German sample.
3) Describe the key dependent variable(s) specifying how they will be measured. - SJT on personal initiative (Bledow & Frese, 2009)
- Personal initiative rating scale (Frese et al., 1997)
- Applicant perceptions of this SJT (see Schäpers et al., 2020): Face validity, perceived predictive validity, perceived knowledge of results, positive affect (Smither et al., 1993); perception of chance to perform (Bauer et al., 2001); test-taking motivation (Arvey et al., 1990)
4) How many and which conditions will participants be assigned to? We aim to collect data at two measurement points: First, in Cuba; then, in Germany.
In each context (Cuba or Germany), participants will be randomly assigned to one of two conditions: They will complete the SJT either with situation description or without situation description.
5) Specify exactly which analyses you will conduct to examine the main question/hypothesis. For evaluating the factor structure, we will conduct a confirmatory factor analysis (CFA). We will use statistical χ2-tests for the comparison between alpha coefficients. Furthermore, we will perform multiple group measurement invariance analyses. For mean comparisons between the German and Cuban sample, as well as between conditions, we will conduct mean comparisons (t-tests, ANOVA).
6) Describe exactly how outliers will be defined and handled, and your precise rule(s) for excluding observations. Following the recommendations of Meade and Craig (2012), we use instructed response items to ensure data quality. Participants' data will be excluded if they fail to answer one of these items correctly and if they indicate at the end of our study that their data should not be used.
7) How many observations will be collected or what will determine sample size? No need to justify decision, but be precise about exactly how the number will be determined. We aim to assess 360 participants, i.e., 180 persons in each cultural context: We conducted an a-priori power analysis in G*Power (Faul et al., 2009) with small to medium effect size (f = 0.175), 4 groups (2 cultural contexts, 2 SJT-conditions), α = .05, and 1-β = .80.
Participants will be recruited from the Universidad de La Habana (Cuba) and the University of Münster (Germany).
8) Anything else you would like to pre-register? (e.g., secondary analyses, variables collected for exploratory purposes, unusual analyses planned?) We will explore differences in test-taking strategies by asking a smaller subsample of participants to participate in in-depth open questionnaires concerning their test-taking strategies.
