Development and empirical user-centered evaluation of semantically-based query recommendation for an electronic health record search engine

https://doi.org/10.1016/j.jbi.2017.01.013Get rights and content
Under an Elsevier user license
open archive

Highlights

  • A user-centered evaluation is conducted to assess the value of query recommendation.

  • The feature is designed to facilitate retrieval of information from EHRs.

  • The algorithm utilizes MetaMap to identify medical concepts.

  • The performance is rated consistently higher with query recommendation turned on.

  • Perceived usefulness and perceived ease of use scores are overwhelmingly positive.

Abstract

Objective

The utility of biomedical information retrieval environments can be severely limited when users lack expertise in constructing effective search queries. To address this issue, we developed a computer-based query recommendation algorithm that suggests semantically interchangeable terms based on an initial user-entered query. In this study, we assessed the value of this approach, which has broad applicability in biomedical information retrieval, by demonstrating its application as part of a search engine that facilitates retrieval of information from electronic health records (EHRs).

Materials and Methods

The query recommendation algorithm utilizes MetaMap to identify medical concepts from search queries and indexed EHR documents. Synonym variants from UMLS are used to expand the concepts along with a synonym set curated from historical EHR search logs. The empirical study involved 33 clinicians and staff who evaluated the system through a set of simulated EHR search tasks. User acceptance was assessed using the widely used technology acceptance model.

Results

The search engine’s performance was rated consistently higher with the query recommendation feature turned on vs. off. The relevance of computer-recommended search terms was also rated high, and in most cases the participants had not thought of these terms on their own. The questions on perceived usefulness and perceived ease of use received overwhelmingly positive responses. A vast majority of the participants wanted the query recommendation feature to be available to assist in their day-to-day EHR search tasks.

Discussion and Conclusion

Challenges persist for users to construct effective search queries when retrieving information from biomedical documents including those from EHRs. This study demonstrates that semantically-based query recommendation is a viable solution to addressing this challenge.

Keywords

Information retrieval systems (L01.700.508.300)
Search engine (L01.470.875)
Electronic health records (E05.318.308.940.968.625.500)
Unified Medical Language System (L01.453.245.945.800)
Query recommendation
Query expansion

Cited by (0)

1

Present address: 6095 Donald Bren Hall, Irvine, CA 92697, USA.