A systematic review of technologies and standards used in the development of rule-based clinical decision support systems

CDSSs fall under two main categories with regards to how they capture medical knowledge [5‐7] that have different approaches on how to infer clinical knowledge (see Fig. 1):

Knowledge-based systems, also known as expert systems or rule-based systems, use an extensive clinical knowledge base containing subject-specific knowledge taken from the medical literature, or from one or more experts. Non-knowledge based systems, also known as case-based systems, use machine learning (ML) to look for patterns in clinical assessment data to suggest a diagnostic probability. They are described as non-knowledge based, as the system does not actually have any knowledge prior to a ML algorithm process taking place.

Non-knowledge approaches are very important to the future of CDSSs as they simplify the knowledge acquisition and maintenance process, a process that is time consuming and requires considerable human effort [2]. This type of system will be more widely used in the future when “big data” applications are widely used in healthcare to provide potentially more accuracy in diagnosing disease than the average clinician [5].The Watson Health system developed by IBM uses natural language processing and information mining tools to provide clinical guidance [8]. Although this type of system can solve complex decision-making problems, they require large sets of labeled data in order to train their algorithms and generally do not provide the rationale behind the decisions made [8]. A detailed survey of machine learning in CDSSs can be found in [9].

This literature review focuses on knowledge-based CDSSs and examples of such systems are discussed in the Analysis section. Knowledge-based CDSSs are composed of three basic architectural parts, namely the Medical Knowledge Base, the Electronic Health Record, and the Inference Mechanism (reasoning) [2], although the EHR may be a separate system. We briefly discuss these components below.

The EHR stores a collection of patient health data in a secure digital form. These records are shared through a network and include information such as medical history, allergies, immunization status, lab results, and personal statistics. The EHR should accurately capture the most up-to-date health status and history of a patient. Standards have been developed to define how EHR data should be structured, semantically described, and communicated. These standards include HL7 (v2, v3, and FHIR), openEHR, ASTM E1384, CEN’s TC 251, and ISO TC 215. EHRs also often rely on medical terminologies such as SNOMED CT (SCT), LOINC, ICD, RxNorm, and UMLS. We discuss standards and terminologies in the following sections.

Rule-based systems are characterized by their high degree of agility to cater for constantly changing circumstances. Rule-based systems are made up from a rule-base (usually decomposed into rule-sets) and an inference engine that operates on the rules. Imperative for this system paradigm approach to work and be easier to maintain is that although the rules collectively should behave as part of the program, they should be stored as data [10].

One of the first CDSSs called MYCIN was developed in the early 1970s at the University of Stanford. Developers accepted that straightforward algorithms of statistical approaches were unsuitable for healthcare with the underlying knowledge not well understood and disagreement even between clinical experts on how best to treat a patient. The nature of these types of problems led the researchers to use interaction rules to capture knowledge about organisms that can cause infections and the antibiotics appropriate to treat them [5, 11].

Rule-based systems are used to develop CDSSs because of their flexibility and agility, an advantageous approach when working with clinical knowledge that is changing constantly with new discoveries from studies. Furthermore, clinicians are not always confident of the rules that should apply to a specific situation [11]. Rule-based systems essentially capture knowledge with conditions in logical systems, similar to first-order logic. The rule is specified in the form of IF–THEN clauses including logical functions and operations, and can be developed in various rule languages or formats [12].

There are now several rules engines that are used widely in diverse domains such as SEBASTIAN, Jess, Jena, CLIPS, OpenCDS, and iLog [13]. System for Evidence-Based Advice through Simultaneous Transaction with an Intelligent Agent across a Network (SEBASTIAN) is a CDSS framework that uses web services, with a write once use everywhere medical knowledge language. The SEBASTIAN framework also includes a rules engine. Jess is a Java-based rules engine that provides rule-based programming for expert systems based on the Rete algorithm for logic inference. Jess is a superset of the CLIPS object-oriented language for implementing expert systems. Jena, a semantic web framework for Java, can extract and write data to Resource Description Framework (RDF) graphs that represent an abstract model. Jena also supports the Web Ontology Language (OWL). OpenCDS is an open-source, standards-based framework that provides tools and resources to implement a CDSS. C Language Integrated Production System (CLIPS) is an object-oriented language that can be used to build expert systems. iLog is a business rules management system with a rule represented by a statement of logic that is used to make a business decision.

In CDSS, there are some major healthcare standards regarding Clinical Guideline Representation, Patient Information and Medical Terminology as identified in [13] that will be discussed in the following subsections.

The plethora of medical standards and the freedom to loosely follow each standard by vendors has resulted in the requirement for a common unified medical knowledge representation approach that will make integration and information exchange easier. In health informatics, ontologies offer a formal description of health-related domain information and provide a vocabulary for biomedical entities that address the unified medical knowledge representation problem [2, 18, 19]. Ontologies facilitate standardisation, flexibility for change, and therefore promote sharing and reusability of medical knowledge between CDSS systems implemented in different technology and standards [2, 20].

An ontology is part of artificial intelligence that focuses on conceptualizations of objects / knowledge. It encompasses a “representation, formal naming, and definition of the categories, properties, and relations between the concepts, data, and entities that substantiate one, many, or all domains” [21]. Ontologies generally appear as a taxonomical tree of conceptualizations, from very general and domain-independent at the top levels to increasingly domain-specific further down the hierarchy and they construct a vocabulary that represents knowledge by classifying and verifying terminologies [22]. As a domain-independent knowledge representation, an ontology is flexible and dynamic in expanding, managing and integrating knowledge from different sources as well as adapting it to the user’s needs and allows easier knowledge sharing between domain applications. However, there is a large number of ontological languages, which poses a challenge for ontological editors when integrating different ontological languages. It may also be difficult to turn specialist knowledge into an ontology particularly when the knowledge including its relationships are not clearly understood/defined or open to contradictions and misinterpretations or if there is a lack of agreement about synonymy when representing knowledge [23].

Ontologies are used in CDSSs to express complex structures and objects with complicated interconnections and still maintain flexibility and compatibility with other systems. Semantic reasoners are software that provide a higher level of abstraction from the ontology layer and can infer logical outcomes from a set of axioms. Thus, the inference engine usually relies on rules that are expressed in an ontology language. Although ontologies can be time consuming to develop they provide key support for exchange of information.

Amongst many ontology languages, the most commonly used ones are RDF, RDF Schema, OIL, DAML + OIL and OWL (OWL, OWL-DL, OWL Lite) with the main difference between them being the level of control on every concept.

OWL extends RDF to allow more complex relationships between RDF classes and provide more precise constraints. An ontology could be created without using an editor, however, there are ontology editors such as Protégé and Ontoterm that provide an interface and visualiser for non-technical users. A review of semantic web and rule-based inference engines is conducted in [12, 24, 25] and some prototypes have been developed to enable easier knowledge and rule capturing (eg. [18]). Clinical knowledge can be inferred using established rule languages such as RuleML and SWRL.

There are existing medical ontologies available online such as SNOMED CT and LOINC, and there are also ontologies that are domain-specific, for example, in diabetes, there is Diabetes Mellitus Treatment Ontology (DMTO), for cardiovascular there is Cardiovascular Disease Ontology (CVDO) and for stroke there is Stroke Ontology (STO-DRAFT/CVAO).

In this work the focus is on rule-based CDSSs and the aim is to answer the following research questions using a systematic review of the literature:

These research questions are designed to help capture theoretical aspects of CDSSs technologies and standards and the extent to which they are used in implementations of CDSSs, i.e. trying to identify evidence of gaps in theory and practice. For this reason this study only includes papers on CDSSs that discuss the underlying technologies used, how knowledge has been represented, and have been evaluated.

A systematic literature review of the most relevant databases with the following search terms was performed (adjusted to each database’s querying method):

(“Clinical Decision Support” OR “Diagnostic Decision Support” OR “e-health”) AND (“rules engine” OR “inference engine” OR

“calculator” OR “business rule” OR “rule based”)

The search was limited to the time period 2007–2019 in seven of the most relevant research databases, namely: Science Direct, Association Computing Machinery (ACM) digital library, Springer, BioMed, PubMed, Emerald, and Institute of Electrical and Electronics Engineering (IEEE) Explore. The application of the search terms on each of these databases yielded 1731 results (see Table 1) and by looking at the title, abstract, and a quick overview of the paper to ensure it discussed technical aspects of the implementation of the CDSS, including how knowledge had been represented and used, this reduced the total to 341 possibly relevant papers (see Fig. 2 for selection process). A thorough review of the 341 possible relevant articles was then further reduced to 114 relevant papers that were then further filtered to 15 works that included some form of rigorous evaluation of the system. This study is based upon those 15 papers (see Table 1 for breakdown of results by database).

Springer had the most results because the search terms could not be applied to the title and abstract only and the search returned papers that had the search terms present on the full text. The number of relevant papers seems more pertinent for Springer once the filtering and review processes were finished.

The papers were distributed between 5 reviewers. The reviewers were given a list of rules as inclusion criteria and the relevant information that should be extracted from the paper, if present. Inclusion criteria included all papers that used a rule-based approach in the health domain, including studies and reviews. Some of the information to be extracted was the medical area of the work, how the logic was integrated in the tool/study, how the Knowledge was represented and acquired, functionality of the used technology, system architecture, platform security and privacy and relevant accreditation, validation and evaluation of the study, hard coded logic, using an existing engine rule engine, rules engine language, logic editor. This information helped to form a useful and comprehensive view of the field of rules engines in the CDSS domain that to the best of knowledge of the authors has not been done before to this extent.

Inter-rater reliability (IRR) was performed and computed using Krippendorff’s alpha, a well-regarded, flexible measure that accounts for chance agreements [26]. The IRR was measured on a subset of the papers found, five papers from each reviewer were selected at random to be reviewed by others. Based on the 5 reviewers, 25 papers and 95 decisions, a Krippendorff’s alpha of 0.742 was computed which can be accepted as satisfactory [27]. The average pairwise agreement was 78.7%.

This section addresses RQ2: What architectures, technologies and standards have been used to develop rules-based CDSSs? and RQ3: Have the CDSSs been integrated with the organizational EHR?

This subsection addresses RQ4: How has knowledge been represented in rules-based CDSSs? From the papers identified, clinical knowledge representation were implemented in a range of ways. Ontology was the most common knowledge representation method identified in 6 of the selected papers. An ontology is normally used to represent:

From the papers identified, no papers used existing ontologies such as those listed in the Background section earlier and instead created bespoke ontologies to represent the knowledge they required. Zhang et al. [34] developed a unified ontology containing diagnostic knowledge, treatment knowledge and pharmaceutical knowledge. [38] used an ontology to represent patient data in different formats and databases, explicit medical knowledge (available texts, journals, and clinical guidelines) and tacit medical knowledge and experiences of domain experts. Kuo and Fuh [40] created a custom health examination ontology to represent health examination data such as physical examination results and lab data. Sherimon and Krishnan [19] developed the Diabetic Patient Clinical Analysis Ontology to represent all the information required to analyze patient information, compute risk scores and suggest treatment procedures according to given clinical guidelines. Hussain et al. [28] used an ontology to represent data from sensors and information retrieved from social media, activity history, user profile information, clinical data, expert knowledge, and expert recommendations. The Knowledge Repository also contained clinical information using HL7 Arden Syntax. Wunsch et al. [37] created a custom ontology based on the Manchester Triage System protocol and Zhang et al. [35] created custom ontologies of real-time patient data and semantic domain knowledge base containing static CDSS knowledge and executable CDSS rules.

OWL was the most used ontology language identified in this study [19, 28, 35, 37, 38] while one paper did not explicitly specify what ontology language was used [34]. Protégé was used by [35, 37] to create their ontologies and Arden syntax was used by [28] to represent expert knowledge and recommendations.

In terms of knowledge repository, a range of technologies were used. Hussain et al. [28] used Windows Azure and SQL Azure while Wunch et al. [37] and Kuo and Fuh [40] used a MySQL database. Zhang et al. [34] employed Microsoft’s Entity Framework to generate a knowledge base schema from an ontology model, which was then stored in a SQL server database. When parsing the ontologies, Wunch et al. [37] and Kuo and Fuh [40] created a custom module written in Java and Zhang et al. [35] used Apache Jena while the other papers did not provide specific details, which may indicate the parsing was part of their bespoke inference engine.

Apart from the use of ontologies, XML and Common LISP (CLISP) were also used for knowledge representation. Chen and Skjelvik [3] represented the guidelines for clubfoot treatment in a decision tree implemented in XML. Kumar et al. [31] used XML as their rule base format. Serhani et al. [29] used CLISP to implement their rules for Jess inference engine.

Other methods were also used for knowledge representation. Kunhimangalam et al. [33] implemented knowledge as fuzzy rules. Karim et al. [36] used knowledge representation based around fuzzy logic to represent different types of uncertain data related to the signs and symptoms of bronchiolitis, which was then converted into conventional IF–THEN rules. Batista-Navarro et al. [32] adopted knowledge from Algorithms of Common Poisonings Part 1 and represented the knowledge as rules stored as CLIPS files in the file system. Kopanitsa and Semenov [39] developed a knowledge representation language based on the first order predicate logic. Sorenson et al. [30] used a frame to represent knowledge where a frame included a list of findings necessary to make a decision or carry out an action, and a logic or mathematical statement to determine its output.

In terms of clinical outcomes, some papers used real-time patient data while others used test data, either based on existing patient data, published test datasets or randomly generated data. Surprisingly many of the patient studies were relatively small. The largest study was for the OntoDiabetic CDSS [19], which was evaluated with 250 patients with risk of CVD, diabetic nephropathy and hypertension. Other real-time patient studies included the cloud-based CDSS [28] that was evaluated with test data from 100 patients: 20 with type-1 diabetes, 40 with type-2 diabetes mellitus, and 40 with suspicions for diabetes but no specific diagnosis; the adaptive CDSS for patients with diabetes linked to an intelligent continuous monitoring scheme [29] evaluated using 13,650 readings related to BS and BP for 70 patients over a period of 6 months; and the frame-based CDSS for ventilator weaning [30] was evaluated with eight patients.

Studies that used test patient data for evaluation included the belief rule-based CDSS for bronchiolitis [36] which used data from 200 patients with bronchiolitis and compared with that of an expert; the fuzzy CDSS for diagnostic and predictive medical opinions, which used a dataset of 104 nerve studies and the output compared with that of a domain expert; the CDSS for poisoning diagnosis [32] which used 50 test cases, each test case consisting of patient symptoms and the expected poisoning type; the mobile CDSS for congenital clubfoot treatment [3] which used a dataset containing treatment history of 17 patients with congenital clubfoot; the CDSS for ICU that adopted CBR and RBR [31], which used real data from six ICU domains using five tagged cases that were very close to an old patient case; and the CDSS using plausible reasoning mechanisms [38], which used a hepatitis dataset dealing with different degrees of missing data, although no indication was given on the size of the dataset.

Other studies used other datasets including the knowledge translation platform for medication prescribing [34], which examined the number of alerts raised by the system and how many prescriptions were modified based on the alert; the UbiTriagem CDSS for triage [37], which used 30 scenarios that already had a correctly documented triage; and the CDSS to analyze laboratory test results [39], which randomly generated 1,000 reports and asked two pathology experts to independently verify the reports.

Addressing RQ2: What architectures, technologies and standards have been used to develop rules-based CDSSs? In terms of architectures, while there are various models proposed the majority of papers were Standalone and only five were Service-Oriented. None of the included papers used the Integrated or Standards-based model. Standards are very important in the development of systems generally and can lead to faster development times, enhanced reliability, enhanced security and can result in safer software. In particular, despite three decades of development CDSSs still suffer from interoperability issues. Many CDSSs exist as standalone systems or as systems that cannot communicate effectively with other systems [41]. Interoperability standards, such as FHIR, are continuously being developed and improved. These standards are already being utilized in commercial EHR systems and government agencies and medical organizations are supporting and some even mandating the use of these interoperability standards in health systems [42].

From the review it is clear that health standards are not widely adopted, with the exception of HL7. Only 3 of the 15 selected papers used recognised CDSS standards [28, 34, 35]. All the standards used were from HL7 including Arden, GELLO, InfoButton, vMR and CDA. None of the selected papers used FHIR or openEHR. This lack of use of interoperability supports the view of other researchers (eg. [41]).

The cloud also offers a solution to interoperability and cloud systems tend to conform to an open architecture, newer standards, and more flexible connectivity to other systems. Only one of the 15 selected papers [28] used the cloud.

It is argued that mHealth is changing the way healthcare can be delivered beyond the use of apps for medicine, health and fitness and well-being of patients to use of remote monitoring and consultation, telemedicine and patient management and in our case clinical decision support. This change is driven by increased mobile device penetration and capabilities along with growing clinician and patient demand [48]. A mHealth-based CDSS offers the potential to reduce medical errors and improve the quality and efficiency of healthcare [49]. Several exploratory studies have been conducted on the use of mobile devices to deliver eHealth content and some had positive outcomes. But one of the issues with this approach is that the majority of mobile applications on Google Play and the Apple Store are not following any health guidelines and are usually not evidence based [50]. A review in 2014 suggested actual application of such systems to date has been limited [51]. We were interested to determine whether the landscape had changed and more mobile CDSSs were being developed and more evidence of their usefulness was available. In this review, 5 of the 15 selected papers supported mobile/handheld device access ([3, 28, 29, 35, 37]) and, as noted previously, none of these evaluations used RCT or quasi-experimental methodologies so like the 2014 review, evidence remains limited.

Addressing RQ3: Have the CDSSs been integrated with the organizational EHR? CDSSs that have been integrated into EHR platforms have been shown to enhance the quality of clinical care and improve patient outcomes and their acceptance by clinicians can be significantly improved if they are well integrated with the EHRs [43‐46]. The integration of these systems also allows previous patient data to be accessed by the CDSS for its own inferencing needs and to save any data the CDSS may generate to be saved back into the EHR. This helps in avoiding redundant data entry and errors due to missing or incorrect data being entered. The integration between the two systems also has several other advantages such as single log on and consistent user interfaces [47]. Despite these benefits, only 2 of the 15 selected papers developed a CDSS that integrated with an EHR.