Game-based Learning Has Good Chemistry with Chemistry Education: A Three-level Meta-analysis

Yuanyuan Hu, Timothy Gallagher, Pieter Wouters, Marieke van der Schaaf, and Liesbeth Kester

Tweet: Does game-based learning hurt or help learning chemistry?

Audience: Educators; Game-based learning designers; Instructional designers; Students; Parents

Key Points

• Game-based learning (GBL) in chemistry was effective for promoting cognition, retention, and motivation among chemistry learners.

• There is substantial variability in the effect size on cognitive outcomes between studies.

• No study reported emotional outcomes (e.g., enjoyment or boredom).

• There may be the small-study effects, particularly publication bias.

Introduction 

This study aims to answer three questions: 1) is the effect of chemistry game-based learning on cognitive, motivational, and emotional outcomes larger than non-GBL? (Media comparison); 2) do instruction characteristics (activity level of control group such as active versus passive learning, additional instruction such as GBL with debriefing versus GBL as standalone, user grouping such as multiple versus single play, number of game sessions such as multiple versus single session) and methodology characteristics (randomization such as random controlled trial versus quasi-experiment design, sample size, publication source such as published studies versus gray literature including these conference proceedings, assessment type such as closed versus non-closed questions) moderate the effect? (Moderator analysis); and 3) which design features (e.g., add competition or add storyline) improve the effectiveness of chemistry GBL? (Value-added comparison). Using a three-level meta -analysis (considering between- and within-study variance), we combined the results of 34 studies out of 842 studies.

Findings 

Standardized mean difference was adopted as effect size (0.2 = small, 0.5 = medium, and 0.8 = large). Compared with non-GBL, chemistry GBL has a statistically significant medium effect on cognition and retention (i.e., the delayed test performance) and a small effect on motivation. No study reported emotional outcomes (e.g., enjoyment or boredom).

There is a statistically significant larger effect in published studies than in gray literature (publication bias) and in small-scale studies than in large-scale studies (small-study effects). Only 3 studies compared GBL with versus without specific design features (e.g., GBL with competition versus GBL without competition).

Takeaways 

Chemistry GBL is more effective not only for cognition and retention but also more motivating than non-GBL. We suggest practitioners consider implementing GBL in chemistry education. This meta- analysis suggests that GBL may address the unique characteristics of a single subject such as chemistry. To draw a full picture of game-based learning, we call for more evidence on emotion, on the relationships between cognition, motivation, and emotion, on design guidelines that improve chemistry game-based learning, and more GBL meta-analyses on subjects other than chemistry.