Elsevier

Computers in Industry

Volume 65, Issue 1, January 2014, Pages 1-23
Computers in Industry

Review
Interoperability evaluation models: A systematic review

https://doi.org/10.1016/j.compind.2013.09.001Get rights and content

Abstract

Interoperability is defined as the ability for two (or more) systems or components to exchange information and to use the information that has been exchanged. There is increasing demand for interoperability between individual software systems. Developing an interoperability evaluation model between software and information systems is difficult, and becoming an important challenge. An interoperability evaluation model allows knowing the degree of interoperability, and lead to the improvement of interoperability. This paper describes the existing interoperability evaluation models, and performs a comparative analysis among their findings to determine the similarities and differences in their philosophy and implementation. This analysis yields a set of recommendations for any party that is open to the idea of creating or improving an interoperability evaluation model.

Introduction

As a multidimensional concept, interoperability can be viewed from numerous perspectives and approached from various directions [1], [2], [3]. Interoperability is a broad and complex subject. Numerous definitions have been given for interoperability. For instance, the following four definitions of interoperability have been given by IEEE [4], [5]: (1) “The ability of two or more systems or elements to exchange information and to use the information that have been exchanged”; (2) “The capability for units of equipment to work efficiently together to provide useful functions”; (3) “The capability – promoted but not guaranteed – achieved through joint conformance with a given set of standards, that enables heterogeneous equipments, generally built by various vendors, to work together in a network environment”; (4) “The ability of two or more systems or components to exchange and use the exchanged information in a heterogeneous network” [6].

The US Department of Defense has also introduced multiple definitions of interoperability; some of which incorporate the IEEE definitions: (1) “The ability of systems, units, or forces to provide services to and accept services from other systems, units, or forces, and to use the services so exchanged to enable them to operate effectively together” [7]; (2) “The condition achieved among communications-electronics systems or items of communications-electronics systems equipment when information or services can be exchanged directly and satisfactorily between them and/or their users. The degree of interoperability should be defined when referring to specific cases” [8]; (3) (a) “Ability of information systems to communicate with each other and exchange information. (b) Conditions, achieved in varying levels, when information systems and/or their components can exchange information directly and satisfactorily between them. (c) The ability to operate software and exchange information in a heterogeneous network (i.e., one large network comprised of several different local area networks). (d) Systems or programs capable of exchanging information and operating together effectively” [9].

Attaining interoperability requires resolution at several distinct levels. According to Ref. [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], there are four levels of interoperability. The interoperability levels are technical, syntactic, semantic, and organizational interoperability.

  • (1)

    Technical Interoperability is achieved among communications-electronics systems or items of communications-electronics equipment when services or information could be exchanged directly and satisfactorily between them and their users [20], [21]. In referring to specific cases, the interoperability degree must be defined [22], [23], [24]. Technical Interoperability is typically associated with hardware/software components, systems, and platforms that enable machine-to-machine communication. This type of interoperability often focuses on communication protocols and the infrastructure required for those protocols to function [25], [26], [27].

  • (2)

    Syntactic interoperability is defined as the ability to exchange data. Syntactic interoperability is generally associated with data formats. The messages transferred by communication protocols should possess a well-defined syntax and encoding, even if only in the form of bit-tables [25], [28].

  • (3)

    Semantic interoperability is defined as the ability to operate on that data according to agreed-upon semantics [29]. Semantic interoperability is normally related to the definition of content, and deals with the human rather than machine interpretation of this content. Thus, interoperability at this level denotes that a common understanding exists between people regarding the definition of the content (information) being exchanged [25], [30], [31], [32].

  • (4)

    Organizational interoperability pertains to the capability of organizations to effectively communicate and transfer meaningful data (information) despite the use of a variety of information systems over significantly different types of infrastructure, possibly across various geographic regions and cultures [33]. Organizational interoperability relies on the successful interoperability of the technical, syntactic, and semantic aspects [25], [34], [35].

For example in Hospital Information Systems (HIS), interoperability is the ability of medical informatics systems to provide services to or to access services from other medical informatics systems and use the services to operate effectively together. If interoperability is evaluated in the healthcare informatics field knowing the strengths and weakness it is possible to make the system more interoperable or to anticipate the potential of interoperability between two or more systems [36]. Another example of interoperability evaluation would be where flight information is passed between the (separate) booking systems for two airlines. Interoperability evaluation would test whether the information reached the target system and still meant the same thing to the target system as the sending system.

Therefore, developing an interoperability evaluation model between software and information systems is difficult, and becoming an important challenge [37], [38], [39], [40], [41], [42], [43]. For this reason, developing and implementing an interoperability evaluation model between software and information systems is extremely problematic [44], [45], [46], [47], [48].

In this direction, this paper presents the existing interoperability evaluation models, and provides an overview of their main concepts and recommendations. Additionally, this paper performs a comparative analysis of the existing interoperability evaluation models to determine the similarities and differences in their philosophy and implementation. This analysis yields a set of recommendations for any party that is open to the idea of creating or improving an interoperability evaluation model.

The structure of the paper is as follows: in the second section the interoperability issues are outlined. An introduction to the available interoperability evaluation models is presented in Section 3. Section 4 compares the interoperability evaluation models under study on the basis of the interoperability issues proposed in Section 2. A discussion on the findings is conducted in Section 5 leading to conclusions in Section 6.

Section snippets

Interoperability issues

This section describes a set of interoperability issues observed in the results of the FP7 ENSEMBLE project [49]. The interoperability issues are categorized into four different granularity levels. The interoperability issues that belong to a higher granularity level are regarded as super-sets of interoperability issues that belong in a lower level [49]. According to the four granularity levels, Fig. 1 illustrates an overview of the identified interoperability issues. Each interoperability

Interoperability evaluation models

Extensive research has been conducted on interoperability evaluation models [50]. This section provides a review on all of the existing evaluation models for interoperability produced since 1980. The interoperability evaluation models were identified through a search of relevant articles published between 1980 and 2012 available on the Web of Science database. Google Scholar was also adopted as a tool to complement the search. Considering their importance, some of the interoperability

Comparative analysis of the interoperability evaluation models

In this section, the indications alongside each interoperability issues in each table denote the corresponding coverage in each particular interoperability evaluation model. Examples are presented below.

  • √ indicates that the interoperability evaluation model has adopted an approach for this criterion, without judging whether this approach provides full or partial coverage for the issue.

  • X refers to the lack of a tangible approach to this issue.

Discussion

The state-of-the-art analysis on the interoperability evaluation models reveals that the existing interoperability evaluation models have carried out considerable efforts to support the interoperability issues in the first, and the second granularity levels, however the interoperability issues in the third granularity level (cloud interoperability) and the fourth granularity level (ecosystems interoperability) are not supported in the existing interoperability evaluation models. Furthermore,

Conclusions

This paper presents an overview of the development of interoperability evaluation models. Several attempts have been made to develop interoperability evaluation models. Interoperability evaluation allows a company to know the strengths and weaknesses to interoperate and to prioritize actions to improve the interoperability.

Based on the findings of this research and the directions provided by relevant literature, future perspectives on the interoperability evaluation models could cover the

Acknowledgement

This research is funded by the University of Malaya Research Grant (UMRG) RG055-11ICT.

Reza Rezaei PhD student in Department of Software Engineering, Faculty of Computer Science and Information Technology, University of Malaya. His research interests are interoperability, cloud computing, and service oriented architecture. He is a lecturer at the Department of Computer Engineering, Faculty of Technical and Engineering Sciences, Islamic Azad University of Saveh Branch, Iran.

References (90)

  • K. Breitfelder et al.

    IEEE 100 the authoritative dictionary of IEEE standards terms

    Standards Information Network IEEE Press

    (2000)
  • J. Radatz et al.
    (1990)
  • A. Geraci et al.

    IEEE Standard Computer Dictionary: Compilation of IEEE Standard Computer Glossaries

    (1991)
  • J. Pub
    (2001)
  • D. Defense

    Chairman of The Joint Chiefs of Staff Instruction, Policy (CJCSI 2800.01C)

    (2001)
  • U.J.F. Command

    Capstone Requirements Document: Global Information Grid (GIG)

    (2001)
  • M. Kasunic et al.

    Measuring Systems Interoperability: Challenges and Opportunities. Technical Note

    (2004)
  • C.E. de Normalisation et al.

    CEN/ISO 11354-1, Part 1: Framework for Enterprise Interoperability

    (2011)
  • D. Chen

    Enterprise interoperability framework

  • E.J. Morris et al.

    System of Systems Interoperability (SOSI): Final Report

    (2004)
  • D. Carney et al.

    Integration and interoperability models for systems of systems

  • S. Munk

    An analysis of basic interoperability related terms, system of interoperability types

    Academic and Applied Research in Military Sciences

    (2002)
  • S. Heiler

    Semantic Interoperability

    ACM Computing Surveys

    (1995)
  • L. Levine et al.
  • M. Novakouski et al.

    Interoperability in the e-Government Context

    (2012)
  • Y. Charalabidis et al.

    Enhancing Application Interoperability and Enabling B2B Transactions over the Internet for Small and Medium Enterprises: The PRAXIS Project CAiSE Workshops (3)

    (2004)
  • T. Kinder

    Mrs. Miller moves house: the interoperability of local public services in Europe

    Journal of European Social Policy

    (2003)
  • K. Kosanke

    ISO Standards for interoperability: a comparison

  • Y. Charalabidis et al.

    Achieving enterprise application interoperability: design patterns and directives, Workshop on Ontology and Enterprise Modelling: Ingredients for Interoperability

    (2004)
  • H. Van der Veer et al.

    Achieving Technical Interoperability – the ETSI Approach

    ETSI White Paper

    (2008)
  • F. Lampathaki et al.

    Infusing scientific foundations into enterprise interoperability

    Computers in Industry

    (2012)
  • Y. Charalabidis et al.

    A review of electronic government interoperability frameworks: patterns and challenges

    International Journal of Electronic Governance

    (2010)
  • Y. Charalabidis et al.

    Enabling interoperability of transactional enterprise applications

  • G.A. Lewis et al.

    Model Problems in Technologies for Interoperability: Web Services (CMU/SEI-2006-TN-021)

    (2006)
  • J. Hall et al.

    Semantic Interoperability for E-Business in the ISP Service Domain, ICE-B

    (2008)
  • Y. Charalabidis et al.

    Towards a scientific foundation for interoperability

    Interoperability in Digital Public Services and Administration: Bridging E-Government and E-Business

    (2010)
  • Y. Charalabidis et al.

    Enterprise Interoperability Research Roadmap

    (2008)
  • G. Gionis et al.

    Enabling cross-organizational interoperability: a hybrid e-business architecture

    Enterprise Interoperability II

    (2007)
  • Y. Charalabidis et al.

    Interoperability approaches for enterprises and administrations worldwide

    The Electronic Journal for E-Commerce Tools and Applications (eJeta)

    (2008)
  • M. Vida et al.

    Measuring medical informatics systems interoperability using the LISI model

  • C. Legner et al.

    Towards an excellence framework for business interoperability

  • E. Commission

    European interoperability framework for pan-european egovernment services

    IDA Working Document, Version 2

    (2004)
  • F. Lillehagen et al.

    ATHENA A1 deliverables, Deliverable DA1. 5.1: MPCE Specification

    (2004)
  • A. Zutshi

    Framework for a Business Interoperability Quotient Measurement Model

    (2010)
  • J. Staff

    Joint Publication 1-02, DOD Dictionary of Military and Associated Terms

    (2008)
  • Cited by (0)

    Reza Rezaei PhD student in Department of Software Engineering, Faculty of Computer Science and Information Technology, University of Malaya. His research interests are interoperability, cloud computing, and service oriented architecture. He is a lecturer at the Department of Computer Engineering, Faculty of Technical and Engineering Sciences, Islamic Azad University of Saveh Branch, Iran.

    Thiam Kian Chiew obtained both his bachelor and masters degrees in computer science from the University of Malaya in 1998 and 2000, respectively. He received his PhD degree in computing science from the University of Glasgow in 2009. He is now a senior lecturer at the Faculty of Computer Science and Information Technology, University of Malaya, Malaysia. His research interests include web engineering, software architecture, and human computer interaction.

    Sai Peck Lee is a professor at Faculty of Computer Science and Information Technology, University of Malaya. She obtained her Master of Computer Science from University of Malaya, her Diplome d’Etudes Approfondies (D.E.A.) in Computer Science from Universite Pierre et Marie Curie (Paris VI) and her Ph.D. degree in Computer Science from Universite Pantheon-Sorbonne (Paris I). Her current research interests in Software Engineering include Object-Oriented Techniques and CASE tools, Software Reuse, Requirements Engineering, Application and Persistence Frameworks, Information Systems and Database Engineering. She has published an academic book, a few book chapters as well as more than 100 papers in various local and international conferences and journals. She has been an active member in the reviewer committees and program committees of several local and international conferences. She is currently in several Experts Referee Panels, both locally and internationally.

    Zeinab Shams Aliee Received Master degree in Software Engineering from Faculty of Computer Science and Information Technology, University of Malaya in 2012. She obtained her Bachelor degree in software engineering from the Azad University of Tehran North Branch, Iran.

    View full text