Skip to content

Systematic literature review

All academic projects should include a review of current literature in order to provide the Reader with sufficient context to understand the detail of the project itself. The purpose of a systematic literature review (SLR) is very different, however. Instead of providing background information, an SLR treats a body of written material as a data source in its own right which it exploits to answer one or more defined research questions. Like any other research methods, an SLR needs to be carefully planned and executed so that the results of the research are credible.

When to use

SLR is the appropriate method when a research question can be answered by interrogating the literature. For example, if the objective is to pin down an objective definition of a developing technological area, an SLR could be used to identify its core characteristics by identifying trends or correlations in published articles. Alternatively, an examination of the literature could help to identify the most effective technical approach to solving a particular class of problem. In this type of research, each published article constitutes a single data point and the review amounts to a statistical examination of the whole dataset.

Kitchenham and Charters (2007) give three reasons for doing and SLR:

  • To summarise the existing evidence concerning a treatment or technology e.g. to summarise the empirical evidence of the benefits and limitations of a specific agile method.
  • To identify any gaps in current research in order to suggest areas for further investigation.
  • To provide a framework/background in order to appropriately position new research activities.

Process

An SLR is characterised by a rigorous approach which may not be present in an ordinary literature review. Without a defined and controlled process, the review cannot claim to be systematic and the results and conclusions cannot be trusted. A project that centres on an SLR should take care to identify and apply recognised structures and techniques as it would with any other research method.

Kitchenham and Charters (2007) divide the process of conducting an SLR into three main stages which are captured diagrammatically in Fig. 1 (Xiao and Watson, 2019):

SLR process

Figure 1: SLR process

Kitchenham and Charters provide guidelines on all the steps mentioned here as do Xiao and Watson. The main thing to bear in mind is that any academic study should be repeatable and therefore the methods used should be explicitly described. Okoli (2017) emphasises this point using a slightly different SLR process.

One of the key parts of an SLR is the definition of a research protocol, and one reason for that is to demonstrate that the process is free from bias. Having a set of rules for selecting literature, and providing evidence of sticking to those rules is important if your Reader is to believe your conclusions. For discussion and guidelines on developing a research protocol, see Drucker et al. (2016) and Keenan (2018).

The British Medical Association defines a 27-step process for conducting an SRL which is summarised in Fig. 2.

BMA SLR process

Figure 2: BMA SLR process

Examples

As well as articles such as those cited above which provide explicit guidance on conducting an SLR, looking at examples can be a great source of inspiration in terms of configuring the process to suit your particular project and in terms of presenting the results. Very often, SLRs are easy to find in Google Scholar because their titles end with '...: a systematic review' but this is not a hard and fast rule. Here are a few to start with:

  • Security Challenges in Software Engineering for the Cloud: A Systematic Review

    The authors introduce a systematic review of articles in the area of software engineering security challenges on the cloud. The review examines articles that were published between 2014 and 2019. The procedure for article qualification relied on the elucidation of Preferred Reporting Items for Systematic Reviews and Meta-Analyses premises. Meta-analysis checklist was employed to explore the analytical quality of the reviewed papers.

  • Advances in Deployment and Orchestration Approaches for IoT - A Systematic Review

    This paper presents the results of an SLR on research approaches and tools for the deployment and orchestration of IoT systems. From thousands of relevant publications, 17 primary studies were systematically identified and reviewed for data extraction and synthesis to answer the predefined research questions.

  • Machine learning applications in urban building energy performance forecasting: A systematic review

    Focusing on an urban-scale, this study systematically reviews 70 journal articles, published in the field of building energy performance forecasting between 2015 and 2018. The recent literature have been categorized according to five criteria: 1. Learning Method, 2. Building Type, 3. Energy Type, 4. Input Data, and 5. Time-scale.