Future directions for depicting credibility in health care web sites

doi:10.1016/j.ijmedinf.2005.07.035

International Journal of Medical Informatics

Volume 75, Issue 1, January 2006, Pages 58-65

https://doi.org/10.1016/j.ijmedinf.2005.07.035 Get rights and content

Summary

Objectives

The purpose of the paper was to determine appropriate terminology, criteria, implementation, and develop a theoretical framework by which credibility in health care web sites can be depicted. Future research directions for evaluating credibility in health care web sites was also discussed.

Method

A comprehensive literature review of published articles, policy papers, and grey literature using relevant search terms was conducted. Sources for articles reviewed included MEDLINE (from 1966 to present), PsycINFO (from 1840 to present), ERIC (from 1966 to present), and the Association for Computing Machinery (ACM) databases. The Web of Science citation service was continuously searched using a subscription service from 2002 to 2004. The search engine Google was also implemented.

Results

A common term, credibility, was purposed for use in this context. A comprehensive set of credibility criteria, presented within a theoretical framework was also developed. Implementation by means of a browser-based graphic icon was purposed.

Conclusions

Relevancy and readiness of the purposed common terminology, criteria, and implementation within the theoretical framework must be further researched. More knowledge of consumers’ behaviour regarding use of online health content and collaboration with others when implementing such information should be considered in future research.

Introduction

Despite various calls for research regarding issues with accuracy in health care material on the world wide web (web) made years ago [1], [2] progress on this issue has been limited. Thus far, a variety of terms intended to describe this issue, including quality, trustworthiness, and credibility have been used. As definitions and interpretations of these terms may vary, operationalization of this concept for research and implementation has been difficult. Further, a wide variety of credibility indicators and guidelines (including those by Wyatt [3], Winker et al. [4], and Silberg et al. [5] as examples) all contained different criteria. Recent research on the usability of these checklists by consumers has found such tools lacking [6]. Further, other research conducted on individuals with lower educational levels, incomes, and literacy levels found their ability to assess credibility was less effective when compared to experts [7].

Concise credibility criteria, compiled by means of a systematic review of relevant literature, provided such indicators but the authors of this review called for future work to find consensus on terminology [8]. In addition, various ways in which criteria should be assessed (either by a third party accredited organization or by consumer themselves) and lack of standardized means of depiction, such as a universally accepted graphic image may have hampered progress in this area. A consistent means to succinctly express credibility in health care web sites has yet to be developed. A first step in resolving these issues is to determine common terminology, criteria, means of implementation, and a theoretical framework from which to conduct research within. This paper proposed solutions to address these four issues. It is hoped that resolution of these issues will also stimulate further research in this important field.

Section snippets

Literature review

Literature searches were conducted at various times throughout 2002 to the present within a variety of databases. For material related to credibility in health care, the MEDLINE (from 1966 to present), PsycINFO (from 1840 to current), and ERIC (from 1966 to present) were searched using the terms ‘world wide web’, ‘WWW’, ‘Internet’, ‘web site’, ‘evaluation’, ‘quality’, ‘trust’, and ‘credibility’. These same terms were also used to search the Association for Computing Machinery (ACM) database to

Moving towards common terminology and criteria

Investigations should, therefore, be conducted investigating the impact and understanding of presenting a web credibility site review using the common terminology, criteria, and implementation within a theoretical framework as outlined in this paper. Participants in research in this area should be instructed on the definition of credibility in study context to ensure appropriate interpretation and subsequent accurate measure. Also necessary is for participants to be consumers of health care

Discussion

Despite an early publication about misinformation appearing online [23], widespread reports of consumers suffering detrimental health from applying treatment information from the web have not been forthcoming [24]. However, lack of reports of death or grievous bodily harm caused by such content may be because attribution of the source is difficult to prove. For example, some deaths are labelled suicide when in fact they are death by misadventure or vice versa. Many treatments referred to on the

Acknowledgement

The research reported herein is partially supported by a CHSRF/CIHR Research Training Award.

References (31)

E.V. Bernstam et al.
Instruments to assess the quality of health information on the world wide web: what can our patients actually use?
Int. J. Med. Inf.
(2005)
M. Bonati et al.
Quality on the Internet
BMJ
(1998)
G. Eysenbach et al.
Towards quality management of medical information on the Internet: evaluation, labelling, and filtering of information
BMJ
(1998)
J. Wyatt
Commentary: measuring quality and impact of the world wide web
BMJ
(1997)
M. Winker et al.
Guidelines for medical and health information sites on the Internet: principles governing AMA web sites
JAMA
(2000)
W. Silberg et al.
Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor—let the reader and viewer beware
JAMA
(1997)
E.G. Benotsch et al.
HIV-AIDS patients’ evaluation of health information on the Internet: the digital divide and vulnerability to fraudulent claims
J. Consult Clin. Psychol.
(2004)
G. Eysenbach et al.
Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review
JAMA
(2002)
Canadian Health Network, How to find the most trustworthy health information web sites online, May 23, 2004, online...
J. Shon et al.
The impact of displayed awards on the credibility and retention of Web site information
Proc. AMIA Symp.
(2000)

B. Fogg et al.

The elements of computer credibility

A.R. Jadad et al.

Rating health information on the Internet: navigating to knowledge or to Babel?

JAMA

(1998)

P. Kim et al.

Published criteria for evaluating health related web sites: review

BMJ

(1999)

P. Stone

Thematic text analysis: new agendas for analysing text content

B. Fogg, Persuasive Technology, Morgan Kaufmann, San Francisco,...

Cited by (34)

Developing Digital Literacies in Undergraduate Nursing Studies: From Research to the Classroom
2017, Health Professionals' Education in the Age of Clinical Information Systems, Mobile Computing and Social Networks
Undergraduate students in the health professions such as nurses are accessing health information from the Internet and social media to inform decisions made in health care practice. Only a small percentage of students are able to adequately appraise the quality of the content of online health information, when they enter their undergraduate programs and this competency develops in the course of educational studies. To increase the competency of nursing students to search, evaluate, and use online health information appropriately as part of nursing practice and health care, nurse educators are challenged to develop strategies and integrate opportunities to teach, mentor, and have students practice informatics skills in undergraduate nursing curricula. The focus on information search and appraisal skills for nurses is emphasized in much of this chapter. Ongoing research may deepen the understanding of effective ways to increase informatics competencies in undergraduate nursing students and lead to a greater understanding of the use of the Internet and social media on health care and patient outcomes.
Preface
2017, Health Professionals' Education in the Age of Clinical Information Systems, Mobile Computing and Social Networks
Exploring factors influencing Chinese user's perceived credibility of health and safety information on Weibo
2015, Computers in Human Behavior
Citation Excerpt :
Thus, the type and extremity of the claims were less likely to influence the users’ judgments of credibility. Showing author qualifications and certifications has been emphasized in guidelines for improving the credibility for online information (Fogg et al., 2003) and for online health information (O’Grady, 2006). Despite the convenience of locating the credential badge of sources on microblog sites, to our surprise, microblog users rarely base their credibility judgment on such information.
The spread of non-credible health and safety information on microblog sites may lead to serious consequences when people use such sites as the basis for critical decisions. This study investigated the factors influencing Chinese microblog users’ perception of the credibility of health and safety information. Credibility cues related to the source (source credentials), the message (claim extremity and claim type), and the distribution in personal networks (type of comments, source of comments, and the number of reposts of a message) were examined. Three experiments were conducted on a mocked up Weibo system with 80 participants. The results show that objective claims with low extremity increased perceived information credibility when the participants were highly involved with the issue and had enough prior knowledge. When the participants had insufficient prior knowledge, the source credentials positively influenced the information’s credibility. Negative comments from personal networks decreased perceived credibility significantly, and this effect was slightly more pronounced when the comments came from close friends. For credible information, a large number of reposts added to the credibility, whereas for less credible information, a large number of reposts may induce greater skepticism and decrease perceived credibility.
Quality control of information on health-related content websites goes further on the internet. Response
2012, Revista Espanola de Cardiologia
The use of tags and tag clouds to discern credible content in online health message forums
2012, International Journal of Medical Informatics
Citation Excerpt :
Authors of early research on online health information-seeking by patients noted there were issues with the reliability and validity of the instruments used to access the credibility of the content found online [13] and subsequently models by which users could assess this were developed [22]. More recently, it has been suggested that users may determine the credibility of information by discussing it in online forums [17]. The new emphasis on collaboration and collective thinking has disrupted linear source-to-user relationships.
Web sites with health-oriented content are potentially harmful if inaccurate or inappropriate medical information is used to make health-related decisions. Checklists, rating systems and guidelines have been developed to help people determine what is credible, but recent Internet technologies emphasize applications that are collaborative in nature, including tags and tag clouds, where site users ‘tag’ or label online content, each using their own labelling system. Concepts such as the date, reference, author, testimonial and quotations are considered predictors of credible content. An understanding of these descriptive tools, how they relate to the depiction of credibility and how this relates to overall efforts to label data in relation to the semantic web has yet to emerge.
This study investigates how structured (pre-determined) and unstructured (user-generated) tags and tag clouds with a multiple word search feature are used by participants to assess credibility of messages posted in online message forums. The targeted respondents were those using web sites message forums for disease self-management. We also explored the relevancy of our findings to the labelling or indexing of data in the context of the semantic web.
Diabetes was chosen as the content area in this study, since (a) this is a condition with increasing prevalence and (b) diabetics have been shown to actively use the Internet to manage their condition. From January to March 2010 participants were recruited using purposive sampling techniques. A screening instrument was used to determine eligibility. The study consisted of a demographic and computer usage survey, a series of usability tests and an interview. We tested participants (N = 22) on two scenarios, each involving tasks that assessed their ability to tag content and search using a tag cloud that included six structured credibility terms (statistics, date, reference, author, testimonial and quotations). MORAE Usability software (version 3.1) was employed to record participants’ use of the study environment. The surveys were analyzed using SPSS version 17. Interviews with participants were transcribed, coded and analyzed using thematic text analysis with the aid of NVivo8.
Most participants had experience with Internet resources. However, less than one quarter of this sample had seen or used tags or a tag clouds. The ways in which participants used tags to label the content posted in the message forums varied. Some participants were tagging the information for their own subsequent use, whereas others viewed this process from the perspective of others: they tagged the content in ways that they thought other users would find beneficial. Many participants did not use the structured credibility tags when asked to search for credible content. The interviews corroborated these findings by confirming participants were not considering credibility foremost when tagging.
Many participants in this study focused on assessing whether the information was relevant to their current circumstances, after which they would proceed to determine its credibility by corroborating with other sources. The use of structured tags to label information may not be a useful way to encourage the use of tagging, or to indicate credibility in this context. Current applications used in the semantic web automate this process. Therefore it may be useful to engage consumers of online content, in particular health-related content, to be more directly involved in the annotation of this content.
The quality of mental disorder information websites: A review
2011, Patient Education and Counseling
Citation Excerpt :
It is likely that such an approach would be more useful if there were a consistent, universal and well-recognised way of indicating high quality websites. A number of other technological solutions have been proposed, for example, the use of web browser encryption icons, and automated quality assessments, but these face significant barriers to implementation [50,51]. In an attempt to assess the effect of feedback on website quality improvement, Jorm et al. [36] scored 52 suicide prevention websites against expert consensus guidelines.
This paper reviews studies assessing the quality of websites providing information about mental disorders.
The review included 31 articles identified by searching research databases in March 2010. Topics covered included affective disorders, anxiety disorders, eating disorders, substance use disorders and schizophrenia/psychosis.
The largest number of articles (13) reported studies assessing affective disorder information quality. Methodologies varied in site selection and rating methods, with some of limited validity. Most concluded that quality was poor, although quality of affective disorder sites may be improving.
There is currently very little understanding of the influence of website quality on user behaviour. Future quality assessments might use the criteria informed by key behaviour change theories.
A possible approach to research on websites and user behaviour might be to develop an evaluation framework incorporating strategies from behaviour change models, key mental health literacy elements and health outcomes relevant to mental health promotion.

View all citing articles on Scopus

View full text

Future directions for depicting credibility in health care web sites

Summary

Objectives

Method

Results

Conclusions

Introduction

Section snippets

Literature review

Moving towards common terminology and criteria

Discussion

Acknowledgement

Int. J. Med. Inf.

Quality on the Internet

BMJ

Towards quality management of medical information on the Internet: evaluation, labelling, and filtering of information

BMJ

Commentary: measuring quality and impact of the world wide web

BMJ

Guidelines for medical and health information sites on the Internet: principles governing AMA web sites

JAMA

Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor—let the reader and viewer beware

JAMA

HIV-AIDS patients’ evaluation of health information on the Internet: the digital divide and vulnerability to fraudulent claims

J. Consult Clin. Psychol.

Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review

JAMA

The impact of displayed awards on the credibility and retention of Web site information

Proc. AMIA Symp.

The elements of computer credibility

Rating health information on the Internet: navigating to knowledge or to Babel?

JAMA

Published criteria for evaluating health related web sites: review

BMJ

Thematic text analysis: new agendas for analysing text content