Skip to main content
Log in

Assessment of osteoporosis-website quality

  • Original Article
  • Published:
Osteoporosis International Aims and scope Submit manuscript

Abstract

Introduction

The Internet provides great opportunities for patient healthcare education, but poses risks that inaccurate, outdated, or harmful information will be disseminated. Osteoporosis is a topic of great interest to patients, many of whom use the Internet to obtain medical information. The aim of this study was to develop and evaluate measurement tools to determine the quality of osteoporosis websites for patients.

Methods

Quality indicators in the categories of content, credibility, navigability, currency, and readability were incorporated into separate evaluation tools for healthcare professionals and for patients. Websites were selected from an Internet search. Interobserver reliability and validity were assessed, and a sample of osteoporosis websites was evaluated by an osteoporosis nurse educator and compared to patient evaluations.

Results

For the quality indicators, there was 79% agreement between the osteoporosis nurse educators, 88% agreement between the physician osteoporosis experts, and 71% agreement comparing the osteoporosis nurse educators to the physician osteoporosis experts. Quality scores for evaluated websites ranged from 18–96 (maximum possible=100), with a mean of 66. Websites with Uniform Resource Locator (URL) suffix .com scored significantly lower compared to those with .gov (P<0.05), .edu (P<0.01), and .org (P<0.01). Healthcare professionals and patients were in agreement on the quality of the highest-rated websites, with less agreement for lower-rated websites.

Conclusions

In summary, a tool for measuring the quality of medical websites was developed and evaluated. Significant variability in osteoporosis-website quality was observed. Higher-quality scores were associated with a higher level of search engine match and specific URL suffixes. A validated tool for evaluating medical websites may have value in assisting patients to select high-quality osteoporosis educational information on the Internet, and may encourage website developers to improve the quality of information that is provided.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  1. Ryan M, Smith KM (2003) Webographies: a patient’s guide to health information on the Internet. Am J Health Syst Pharm 60:876–877, 881

    PubMed  Google Scholar 

  2. Krane D Healthcare News (2005) Number of “cyberchondriacs”- US adults who go online for health information- increases to estimated 117 million. http://www.harrisinteractive.com/news/newsletters/healthnews/HI_HealthCareNews2005Vol5_Iss08.pdf. Cited 21 Aug 2005

  3. Eysenbach G, Powell J, Kuss O, Sa ER (2002) Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA 287:2691–2700

    Article  PubMed  Google Scholar 

  4. Jain T, Barbieri RL (2005) Website quality assessment: mistaking apples for oranges. Fertil Steril 83:545–547

    Article  PubMed  Google Scholar 

  5. Huang JY, Discepola F, Al-Fozan H, Tulandi T (2005) Quality of fertility clinic websites. Fertil Steril 83:538–544

    Article  PubMed  Google Scholar 

  6. RAND Health (2001) Proceed with caution: a report on the quality of health information on the Internet. http://www.chcf.org/documents/consumer/ProceedWithCautionCompleteStudy.pdf. Cited 7 Sept 2005

  7. Benigeri M, Pluye P (2003) Shortcomings of health information on the Internet. Health Promot Int 18:381–386

    Article  PubMed  Google Scholar 

  8. Walji M, Sagaram S, Sagaram D, Meric-Bernstam F, Johnson C, Mirza NQ, Bernstam EV (2004) Efficacy of quality criteria to identify potentially harmful information: a cross-sectional survey of complementary and alternative medicine web sites. J Med Internet Res 6:e21

    Article  PubMed  Google Scholar 

  9. Matthews SC, Camacho A, Mills PJ, Dimsdale JE (2003) The internet for medical information about cancer: help or hindrance? Psychosomatics 44:100–103

    Article  PubMed  Google Scholar 

  10. Green CJ, Kazanjian A, Helmer D (2004) Informing, advising, or persuading? An assessment of bone mineral density testing information from consumer health websites. Int J Technol Assess Health Care 20:156–166

    Article  PubMed  Google Scholar 

  11. URAC (2005) Health web site check-up service and accreditation program. http://www.urac.org/prog_accred_HWS_po.asp?navid=accreditation&pagename=prog_accred_HWS. Cited 7 Sept 2005

  12. Internet Healthcare Coalition (2005) eHealth Code of Ethics. http://www.ihealthcoalition.org/ethics/ethics.html. Cited 21 Aug 2005

  13. Winker M, Flanagin A, Chi-Lum B, White J, Andrews K, Kennett R, DeAngelis C, Musacchio R (2005) Guidelines for medical and health information sites on the Internet. http://www.ama-assn.org/ama/pub/category/1905.html. Cited 27 Aug 2005

  14. Health On the Net Foundation (2005) http://www.hon.ch/. Cited 21 Aug 2005

  15. Risk A, Dzenowagis J (2001) Review of internet health information quality initiatives. J Med Internet Res 3:E28

    Article  PubMed  CAS  Google Scholar 

  16. Gagliardi A, Jadad AR (2002) Examination of instruments used to rate quality of health information on the internet: chronicle of a voyage with an unclear destination. BMJ 324:569–573

    Article  PubMed  Google Scholar 

  17. Jadad AR, Gagliardi A (1998) Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 279:611–614

    Article  PubMed  CAS  Google Scholar 

  18. Sullivan D (2005) Nielsen NetRatings search engine ratings. http://searchenginewatch.com/reports/article.php/2156451.Cited 21 Aug 2005

  19. National Osteoporosis Foundation (2002) America’s bone health: the state of osteoporosis and low bone mass in our nation. National Osteoporosis Foundation, Washington, DC

    Google Scholar 

  20. Greenberg L, D’Andrea G, Lorence D (2003) Setting the public agenda for online health search, a white paper and action agenda. http://www.urac.org/documents/URAC-CWWHealthSearchWhitePaper1203.pdf. Cited 7 Sept 2005

  21. Ambre J, Guard R, Perveiler F, Renner J, Rippen H (1997) Criteria for assessing the quality of health information on the Internet. http://hitiweb.mitretek.org/docs/criteria.pdf. Cited 7 Sept 2005

  22. Ansani NT, Vogt M, Henderson BA, McKaveney TP, Weber RJ, Smith RB, Burda M, Kwoh CK, Osial TA, Starz T (2005) Quality of arthritis information on the Internet. Am J Health Syst Pharm 62:1184–1189

    Article  PubMed  Google Scholar 

  23. Howitt A, Clement S, de LS, Thiru K, Goodwin D, Wells S (2002) An evaluation of general practice websites in the UK. Fam Pract 19:547–556

    Article  PubMed  Google Scholar 

  24. Flesch RF (1951) How to test readability. Harper, New York

    Google Scholar 

  25. Kincaid J, Fishburne R, Rogers R, Chissom B (1975) Derivation of new readability formulas: automated reliability index, fog count, and Flesch reading ease formula for Navy enlisted personnel (branch report 8-75). US Navy, Millington, TN

  26. Doak CC, Doak LG, Root JH (1996) Teaching patients with low literacy skills. J.B. Lippincott, Philadelphia

    Google Scholar 

  27. D’Alessandro DM, Kingsley P, Johnson-West J (2001) The readability of pediatric patient education materials on the World Wide Web. Arch Pediatr Adolesc Med 155:807–812

    PubMed  CAS  Google Scholar 

  28. Williams DM, Counselman FL, Caggiano CD (1996) Emergency department discharge instructions and patient literacy: a problem of disparity. Am J Emerg Med 14:19–22

    Article  PubMed  CAS  Google Scholar 

  29. Lee PP (1999) Why literacy matters. Links between reading ability and health. Arch Ophthalmol 117:100–103

    PubMed  CAS  Google Scholar 

  30. Kenkel PJ (2002) Boon for consumers? Hospital quality regulator and big business health purchasing coop join forces. Manag Care Q 10:51

    PubMed  Google Scholar 

  31. O’Leary DS (1987) Leaders speak out on the quality of healthcare delivery. Interview by Richard L. Clarke. Healthc Financ Manage 41:38–40, 44

    PubMed  CAS  Google Scholar 

  32. Watts NB (2002) Bone quality: getting closer to a definition. J Bone Miner Res 17(7):1148–1150

    Article  PubMed  Google Scholar 

  33. Bernstam EV, Shelton DM, Walji M, Meric-Bernstam F (2005) Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use? Int J Med Inform 74:13–19

    Article  PubMed  Google Scholar 

  34. Fricke M, Fallis D, Jones M, Luszko GM (2005) Consumer health information on the Internet about carpal tunnel syndrome: indicators of accuracy. Am J Med 118:168–174

    Article  PubMed  Google Scholar 

  35. Fraquelli M, Conte D, Camma C, Casazza G, Di BD, Rebulla P, Prati D (2004) Quality-related variables at hepatological websites. Dig Liver Dis 36:533–538

    Article  PubMed  CAS  Google Scholar 

  36. Butler L, Foster NE (2003) Back pain online: a cross-sectional survey of the quality of web-based information on low back pain. Spine 28:395–401

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to E. M. Lewiecki.

Additional information

Prior abstract presentation: Preliminary data for this study were presented as a plenary poster at the 2005 ISCD Annual Meeting in New Orleans, LA, USA, and published as an abstract in Lewiecki EM, Rudolph LA, Kiebzak GM (2005) Evaluation of osteoporosis website quality. J Clin Densitom 8:243

Appendices

Appendix 1

Observer Initials

Website

Instructions: The main page to be evaluated is the “linked” page (LP) generated from the Google search. If LP is not the primary education page (PEP), then PEP must be evaluated as well. In addition to viewing LP and/or PEP, browse home page and other web pages enough to determine whether there are any obvious problems or bias. Each item identifies whether evaluation is for LP, PEP, or entire website (WS). Record score of “0” or “1” in every highlighted box.

Healthcare Website Assessment Tool (HWAT) 3.0

  

Present or absent

Weighting (9,6,4)

Score

  

(0 or 1)

  

Content

    

WS

1. Originator (person or organization) of website identified

 

9

 

LP

2. Subject of website clearly stated in title or description

 

9

 

WS

3. Unbiased- website clearly distinguishes sale of products/services from educational information

 

9

 

WS

4. Website includes definition, consequences, prevention, treatment of osteoporosis (note: linked websites are excluded from analysis)

 

6

 

Credibility

    

LP/PEP

5. Author/institution providing information clearly stated

 

9

 

WS

6. “Seal of Approval” (HONcode, URAC, eCode of ethics, or equivalent)

 

6

 

LP/PEP

7. Credible source (university, professional society, government agency, non-profit foundation) or published references are provided

 

9

 

Navigability

    

WS

8. Functioning intrawebsite links

 

9

 

WS

9. Functioning interwebsite links

 

9

 

LP

10. Fully printable (margins not cut off)

 

9

 

WS

11. Email link for feedback to website provider

 

6

 

Currency

    

LP

12. Revision or copyright in last 12 months (score 0 if not provided)

 

6

 

Readability

    

PEP

13. Flesch-Kincaid Grade Level (score 0 if >8, 1 if 8 or less)=________

 

4

 

Total Score

  

100

 

Appendix 2

If you are age 50 or older and have access to the Internet, you are invited to participate in a study to express your opinions on websites providing osteoporosis educational information for patients. By logging on to ten different websites and completing five multiple choice questions for each, your results will help us to determine whether these websites provide you with helpful and accurate information. The websites may vary widely in their quality. Please be totally honest in your answers. This is not a test. There are no right or wrong answers. After you have completed all the questionnaires, return this page with all forms to:

[address]

You will earn $25 for completing questionnaires for 10 web sites. The check will be written to you or donated in your name to [name of nonprofit osteoporosis foundation], according to your preference.

Please provide the following information about yourself:

  • Sex: Female □ Male □

  • Age:

  • Highest level of education:

    • Less than high school □

    • High school □

    • Degree beyond high school □

  • Computer skills

    • Beginner □

    • Intermediate □

    • Advanced □

Indicate preference for payment of $25 by checking one of the following:

  • □ Mail check to me at this address-

  • □ Donate $25 in my name to [name of nonprofit osteoporosis foundation]

Website: [URL of website]

Circle the answer to each of the following questions for this website.

  1. 1.

    This website offers information about osteoporosis that is helpful to me.

    1. a.

      Agree

    2. b.

      Disagree

    3. c.

      Not sure

  2. 2.

    I believe the information provided on this website is scientifically correct.

    1. a.

      Agree

    2. b.

      Disagree

    3. c.

      Not sure

  3. 3.

    It is easy to find information I need on this website.

    1. a.

      Agree

    2. b.

      Disagree

    3. c.

      Not sure

  4. 4.

    The information on this website is up-to-date.

    1. a.

      Agree

    2. b.

      Disagree

    3. c.

      Not sure

  5. 5.

    The information on this website is easy to understand.

    1. a.

      Agree

    2. b.

      Disagree

    3. c.

      Not sure

Appendix 3

Google search matching number

Qualifying order

URL

Website title

Weighted score

1

1

http://www.nof.org/

National Osteoporosis Foundation

90

3

2

http://www.osteoporosis.ca/

Osteoporosis Society of Canada

96

5

3

http://www.osteo.org/

NIH ORBD-NRC - Osteoporosis and Related Bone Diseases

96

7

4

http://www.nos.org.uk/

National Osteoporosis Society

90

9

5

http://www.osteofound.org/

International Osteoporosis Foundation

87

12

6

http://www.nlm.nih.gov/medlineplus/osteoporosis.html

MedlinePlus: Osteoporosis

96

16

7

http://www.fore.org/

FORE

84

17

8

http://www.endocrineweb.com/osteoporosis/

The Osteoporosis Center

51

20

9

http://www.osteoporosis.org.au/

Osteoporosis Australia

51

24

10

http://www.fda.gov/fdac/features/796_bone.html

Boning Up on Osteoporosis

81

26

11

http://imaginis.com/osteoporosis/

Imaginis - Osteoporosis

48

27

12

http://www.obgyn.net/osteo/osteo.asp

OBGYN.net - Osteoporosis

66

30

13

http://www.medinfo.co.uk/conditions/osteoporosis.html

Medinfo: Osteoporosis

84

31

14

http://www.osteoporosis-centre.org/

Osteoporosis Centre

42

32

15

http://www.crha-health.ab.ca/hlthconn/items/osteop.htm

CHR:YOUR HEALTH

42

33

16

http://www.osteoporosis-support.com/

Stop Osteoporosis!!!

48

38

17

http://www.4woman.gov/faq/osteopor.htm

Osteoporosis

69

40

18

http://dir.yahoo.com/Health/Diseases_and_Conditions/ Osteoporosis/

Yahoo! Directory Osteoporosis

51

42

19

http://www.acu-cell.com/dis-ost.html

Osteoporosis/Bone Loss

63

43

20

http://www.betterbones.com/

Osteoporosis Education Project

51

47

21

http://www.medicalresources.org/osteo.html

medicalresources.org

96

50

22

http://courses.washington.edu/bonephys/ophome.html

Osteoporosis and bone physiology

87

53

23

http://orthopedics.about.com/health/orthopedics/ blosteoporosis.htm

Osteoporosis

33

54

24

http://www.osteorec.com/

Osteoporosis Resource Centers

66

55

25

http://www.arc.org.uk/about_arth/booklets/6028/6028.htm

Arthritis Research Campaign: Osteoporosis

66

57

26

http://www.physsportsmed.com/issues/1998/02feb/katzpa.htm

The Physician and Sportsmedicine

78

58

27

http://health.discovery.com/diseasesandcond/ encyclopedia/1666.html

Discovery Health: osteoporosis

51

61

28

http://www.nal.usda.gov/fnic/etext/000107.html

Osteoporosis Topic Page

96

72

29

http://www.osteoporosis-treatment.net/

Osteoporosis Treatment

30

74

30

http://www.creatinghealth.psu.edu/osteo/definition.html

Creating Health- Osteoporosis

81

75

31

http://arthritisinsight.com/medical/disease/op.html

Arthritis Insight- Osteoporosis Information

72

76

32

http://www.arthritis.ca/types%20of%20arthritis/ osteoporosis/default.asp?s=1

Osteoporosis

87

77

33

http://www.aafp.org/afp/990700ap/194.html

Prevention of Osteoporosis

75

80

34

http://www.oif.org/site/PageServer?pagename=Osteo

Osteogenesis Imperfecta Foundation: Osteoporosis

81

81

35

http://www.thebestisyet.net/cgi-bin/cgiwrap/ pando19/start.cgi/viewfrom16.htm

Diagnosis: Osteoporosis

45

82

36

http://www.merck.com/mrkshared/mmanu75al/ section5/chapter57/57a.jsp

The Merck Manual

84

86

37

http://www.project-aware.org/Heal75th/Osteo/osteo-what.shtml

Osteoporosis, osteopenia, bone density

75

90

38

http://hebw.uwcm.ac.uk/osteoporosis/

Osteoporosis

75

94

39

http://www.rcplondon.ac.uk/pubs/wp_osteo_update.htm

Osteoporosis Update

75

96

40

http://www.dietitian.com/calcium.html

Calcium & Osteoporosis

27

97

41

http://www.medicinenet.com/Osteoporosis/article.htm

Osteoporosis

72

99

42

http://www.healingwithnutrition.com/odisease/ osteoporosis/osteoporosis.html

Osteoporosis

57

102

43

http://www.healthology.com/osteoporosis

Healthology Osteoporosis

48

103

44

http://www.actonel.com/

Welcome to Actonel

51

105

45

http://www.mja.com.au/public/guides/osteo/ostindex.html

MJA: Osteoporosis

63

108

46

http://www.womenshealthchannel.com/osteoporosis/index.shtml

Osteoporosis- WomensHealthChannel

96

112

47

http://www.science.org.au/nova/042/042key.htm

Osteoporosis

78

113

48

http://www.medicare.gov/Health/OsteoporosisDetails.asp

Medicare.gov- Osteoporosis Details

51

116

49

http://lancaster.unl.edu/food/ftm-j01.htm

Nutrition and Osteoporosis

78

120

50

http://www.scoi.com/osteo.htm

SCOI Presents Osteoporosis

60

121

51

http://www.osteoporosis-centre.org/ct_osteo.htm

Osteoporosis on the Internet

24

122

52

http://www2.rpa.net/~rcfisher/

Research and recipes

18

124

53

http://www.oznet.ksu.edu/humannutrition/ nutlink/pages/BONES.HTM

Osteoporosis

81

126

54

http://www.netdoctor.co.uk/diseases/facts/osteoporosis.htm

Osteoporosis

69

127

55

http://www.silcom.com/~dwsmith/boned394.html

Postmenopausal Osteoporosis

18

129

56

http://www.spine-health.com/topics/cd/ osteoporosis/osteopor02.html

What you need to know

75

130

57

http://www.aapmr.org/condtreat/other/osteo.htm

AAPM&R

51

132

58

http://www.jr2.ox.ac.uk/bandolier/booth/alternat/AT059.html

Exercise and Osteoporosis

33

133

59

http://www.healthfinder.gov/Scripts/ SearchContext.asp?topic=614

Healthfinder

81

134

60

http://www.coloradohealthsite.org/osteo/osteo_main.html

Colorado HealthSite

78

135

61

http://www.intelihealth.com/IH/ihtIH/WSIHW000/9339/9564.html

InteliHealth: osteoporosis

90

137

62

http://www.orthoteers.co.uk/Nrujp~ij33lm/Orthboneop.htm

Osteoporosis

60

138

63

http://www.cancersupportivecare.com/osteoporosis.html

Osteoporosis

81

139

64

http://hcd2.bupa.co.uk/fact_sheets/html/Osteoporosis.html

Osteoporosis

60

140

65

http://www.tdh.state.tx.us/osteo/default.htm

Texas Dept. of Health

84

141

66

http://www.healthatoz.com/healthatoz/Atoz/ dc/caz/bone/poro/osteoCFindex.html

Osteoporosis

66

143

67

http://www.lef.org/protocols/prtcl-085.shtml

Osteoporosis

57

146

68

http://www.umc.sunysb.edu/internalmed/osteo/osteo.html

Osteoporosis Center

84

147

69

http://www.infobreastcancer.ca/osteo.htm

Osteoporosis

66

148

70

http://www.healthservices.gov.bc.ca/msp/ protoguides/gps/bone/bonepg.html

Bone Density Measurement and Osteoporosis

57

149

71

http://body.subportal.com/health/Womens/Osteoporosis/

aceZones.com

24

153

72

http://www.newscientist.com/news/news.jsp?id=ns99994421

New Scientist

24

154

73

http://en.wikipedia.org/wiki/Osteoporosis

Osteoporosis- Wikipedia

63

161

74

http://www.johnleemd.com/trutabos.html

Truth About Osteoporosis

48

163

75

http://www.menopause-online.com/diseases.htm

Menopause Online

33

174

76

http://www.dundee.ac.uk/medicine/tayendoweb/ images/osteoporosis.htm

Osteoporosis

60

178

77

http://curezone.com/dis/1.asp?C0=257

Osteoporosis Cure

28

179

78

http://www.ridgeway-surgery.demon.co.uk/health/osteopor.htm

The Ridgeway Surgery: osteoporosis

42

181

79

http://www.vegsource.com/articles/calcium_update.htm

VegSource.com

27

183

80

http://qualitycounts.com/fposteoporosis.html

Osteoporosis

48

184

81

http://cme.ufl.edu/media/osteo/

Post-menopausal Osteoporosis

87

185

82

http://health.allrefer.com/health/osteoporosis-info.html

AllRefer Health - Osteoporosis

69

197

83

http://www.sutterhealth.org/ health/healthinfo/index.cfm?page=article&sgml_id=hw131419

Health Information

72

198

84

http://Medic.med.uth.tmc.edu/ptnt/00000767.htm

MEdIC - Health Explorer - Osteoporosis

69

200

85

http://www.vitacost.com/science/hn/Concern/Osteoporosis.htm

Osteoporosis

24

201

86

http://www.roche-diagnostics.com/health_kiosk/ framesets/3.html

Roche Health Kiosk

48

202

87

http://nursing.villanova.edu/womenwithdisabilities/ other/osteoJ.htm

Osteoporosis

57

204

88

http://www.tourismofindia.com/exi/hitosteoporosis.htm

Tourism of India

24

205

89

http://www.healthcentral.com/mhc/top/000360.cfm

HealthCentral

48

206

90

http://www.hormone.org/learn/osteo_2.html

Osteoporosis

66

207

91

http://www.state.vt.us/health/_hs/epidemiology/ osteoporosis/osteoporosis.htm

Osteoporosis

84

208

92

http://www.webhealthcentre.com/centers/osteo.asp

WebHealthCentre.com

66

218

93

http://www.gemedicalsystems.com/rad/bonedens/osteo/osteo.html

Bone Densitometry - Osteoporosis

45

219

94

http://www.ostex.com/ Osteoporosis/Osteoporosis_Related_Links.asp

Osteoporosis Related Links

27

224

95

http://cpmcnet.columbia.edu/texts/gcps/gcps0056.html

Clinical Preventive Services

72

225

96

http://www.thebody.com/sfaf/summer01/osteoporosis.html

The Body: SFAF BETA

69

226

97

http://osteoporosis.upmc.com/

Osteoporosis

81

227

98

http://www.mhcs.health.nsw.gov.au/health-public-affairs/ mhcs/publications/4925.html

How strong are your bones?

90

228

99

http://www.lhj.com/home/Osteoporosis.html

Osteoporosis

58

229

100

http://www.hersource.com/osteo/c1/index.cfm

Osteoporosis

69

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lewiecki, E.M., Rudolph, L.A., Kiebzak, G.M. et al. Assessment of osteoporosis-website quality. Osteoporos Int 17, 741–752 (2006). https://doi.org/10.1007/s00198-005-0042-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00198-005-0042-5

Keywords

Navigation