Systematic Reviews

Team Members: Robert Bernard (Leader), Phil Abrami, Richard Schmid, and Anne Wade

Coordinator:
Evgueni Borokhovski

Members of the Systematic Review theme work in teams to produce systematic reviews of the impact of educational interventions and develop methodologies for systematic review and meta-analysis. A systematic review is a compilation and synthesis of research on a clearly formulated question that may address issues of educational theory, practice, policy or a combination of the three. Using methods that are objective, repeatable, transparent, systematic, comprehensive and integrative, members of our theme identify, select and critically appraise relevant research for inclusion in each review, and collect and analyze data from the included research. If statistical methods are used to synthesize research, the review is generally referred to as a meta-analysis.

Over the past year, we worked on three major systematic review projects: 1) a review of instructional interventions that potentially affect critical thinking skills and dispositions; 2) a meta-analysis of the effects of interaction treatments in distance education; and 3) a synthesis of technology treatments that affect achievement and attitudes in postsecondary students.

These projects yielded two major pieces published in Review of Educational Research, the prestigious review journal of the American Educational Research Association and an article in Journal of Computing in Higher Education.

Abrami, P.C., Bernard, R.M., Borokhovski, E., Wade, A., Surkes, M., Tamim, R., & Zhang, D. A. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage one meta-analysis. Review of Educational Research, 78(4), 1102-1134.
Bernard, R. M., Abrami, P. C., Borokhovski, E., Wade, C. A., Tamim, R., Surkes, M. A., & Bethel, E. C. (2009). A meta-analysis of three types of interaction treatments in distance education. Manuscript in press, Review of Educational Research.
Schmid, R. F., Bernard, R.M., Borokhovski, E., Tamim, R., Abrami, P.C., Wade, C.A., Surkes, M.A., & Lowerison, G. (2009). Technology’s effect on achievement in higher education: A Stage I meta-analysis of classroom applications. Journal of Computing in Higher Education, 41(2), 95-109.

Rana Tamim, a student member of our systematic review team for the last three years, completed her Ph.D. by conducting a second-order meta-analysis of meta-analyses addressing the issue of technology integration in education. In addition, she developed a methodological quality index for assessing meta-analyses.

Four members of the team, Anne Wade, Phil Abrami, Evgueni Borokhovski and Robert Bernard, attended the Campbell Collaboration’s annual colloquium in Oslo, Norway where they conducted two workshops, presented two papers and were involved in a symposium on conducting and disseminating brief reviews. Three prominent members of the systematic review community, Mark Lipsey, Charles Ungerleider and David Gough, also participated in the symposium.

On a final note, an article published in 2004 (Bernard, Abrami, Lou , Borokhovski, Wade, Wozney, Wallet, Fiset & Huang, 2004) entitled, “How does distance education compare to classroom instruction? A meta-analysis of the empirical evidence,” has now been on the Review of Educational Research’s list of the 50 most downloaded/most read articles for four and one-half years. This we believe is a testament to the timeliness and quality of our systematic reviews.