Systematic Literature Review: Implementations of e-Learning

Review Guidelines

This review was split into three main phases: planning the review, conducting the review, and reporting the review. Kitchenham's research into systematic literature reviews provided insight on how to perform each stage. An example of a systematic literature review conducted by her is is found in this paper: A systematic review of systematic review process research in software engineering (Kitchenham & Brereton, 2013). There are some methodical differences intentionally made in my review to take advantage of the context of my review, but I ensured any differences still satisfied Kitchenham's guidelines. The reporting of this review also followed PRISMA guidelines.

Introduction

Objectives

The review initially aimed to answer the following questions:

• RQ1 - What instructional models have shaped implementations of E-learning?
• RQ2 - What design principles have shaped E-learning interfaces?
• RQ3 - What technologies have shaped implementations of E-Learning?
• RQ4 - What has constituted the quality of implementations of E-learning?
• RQ5 - What have been the most described research limitations?

Methods

Eligibility Criteria

Literature needed to meet the following criteria to be eligibile for inclusion in this literature review:

The content of the literature is accessible by satisfying all of the following:

• The content was written in English..
• The content included explicit research questions or aims.
• The content included explicit details of conducted research methodology.

The content of the literature was high quality, indicated by all of the following:

• The literature was published in a peer reviewed journal.
• The literature was published from 2020 onwards.
• The literature was published in a journal that was either recommended in academic literature, or had high peformance indicators.

The content of the literature is relevant by satisfying at least one of the following:

• The content describes and focuses on people interacting with educational technology.
• The content describes and focuses on technology that constitutes and composes educational technology.
• The content describes and focuses on services implemented by educational technology.

Information Sources

The literature search was confined to a set journals that arguably constituted the highest quality journals dedicated to 'educational technology' that were accessible.

The first source of peer-reviewed journals was taken from the paper Open Access Journals in Educational Technology: Results of a Survey of Experienced Users (Perkins & Lowenthal., 2016) . The paper provides a list of top 25 journals that was gathered from a survey completed by 236 scholars, who had all published in open-access journals related to educational technology; and some consideration of journal performance metrics. The credibility of the scholars participating in the journal review, along with the credibility of Lowenthal, who has a h-index of 40, provide plausability to the journals being academically credible, so I've opted to use Perkins & Lowenthal's list of top 25 journals to help filter the literature search. It's worth noting that after searching citations and the authors more recent papers, no other journal recommendation papers were found as alternatives to this list. I believe the continued academic journal review process is beneficial for the systematic development of future literature reviews.

The second source of peer-reviewed journals was a set based upon bibliometric data, specifically I opted to include journals that have high h5-indexes, indicating that in the last 5 years, they have published high impact articles. A list of 20 journals with high h5-indexes is provided by Google Scholar. The h5-index metric was selected as it specifically relates to recent article popularity, and Google Scholar was selected due to the high amount of journals it indexes and it's accessibility. World of Science would have been selected too if it was free to use.

Search Strategy

The first phase of the search involved searching only the most recent issue of each journal. The papers were initially scanned to see if they met all of the eligibility criteria and if so, the paper was shortlisted for the selection process. Progress was tracked using a kanban board. An example of the kanban board is shown below

Selection Process

The selection process made use of automation tools available in the software Notion. Papers were further scrutinised if they met any of the following criteria:

• The paper included an implementation diagram, and had technology as an object of study.
• The paper included an educational assignment as a participant task, and had technology as an object of study.
• The paper included classification or axial coding as a data synthesis method, with context categories including education and technology.

An example of progress during this phase is shown below.

Data Collection Process

The review was conducted by one person manually extracting data from selected articles and storing it on the information management system Notion . A link to the collected data can be found here: The Implementation of E-Learning - A Review Workspace

Results

The final paper can be found here.