Skip to main content
Log in

Open-book Tests to Complement Assessment-programmes: Analysis of Open and Closed-book Tests

  • Published:
Advances in Health Sciences Education Aims and scope Submit manuscript

Abstract

Today’s health sciences educational programmes have to deal with a growing and changing amount of knowledge. It is becoming increasingly important for students to be able to use and manage knowledge. We suggest incorporating open-book tests in assessment programmes to meet these changes. This view on the use of open-book tests is discussed and the influence on test quality is examined. To cope with the growing amount of medical knowledge, we have divided the body of knowledge into core knowledge, which students must know without need for references, and backup knowledge, which students need to understand and use properly with the help of references if so desired. As a result, all tests consist of a subtest for reproduction and understanding of core knowledge (a closed-book test) and a subtest for the ability to understand and manage backup knowledge (an open-book test). Statistical data from 14 such double-subtest exams for first and second-year students were analyzed for two cohorts (N = 435 and N = 449) with multilevel analysis, in accordance with generalizability theory. The reliability of the open and closed-book sections of the separate tests varied between 0.712 and 0.850. The open-book items reduce reliability somewhat. The estimated disattenuated correlation was 0.960 and 0.937 for cohorts 1 and 2 respectively. It is concluded that the use of open-book items with closed-book items slightly decreases test reliability but the overall index is acceptable. In addition, open and closed-book sections are strongly positively related. Therefore, open-book tests could be helpful in complementing today’s assessment programmes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Baillie C., Toohey S. (1997). The power test: its impact on student learning in a materials science course for engineering students. Assessment and Evaluation in Higher Education 22:33–49

    Article  Google Scholar 

  • Boniface D. (1985). Candidates’ use of notes and textbooks during an open-book test. Educational Research 27:201–209

    Article  Google Scholar 

  • Brennan R.L. (2001). Generalizability Theory. Springer-Verlag, New York

    Google Scholar 

  • Broyles I.L., Cyr P.R., Korsen N. (2005). Open book tests: assessment of academic learning in clerkships. Medical Teacher 5: 456–462

    Article  Google Scholar 

  • Cain J.C. (1979). Continuing medical education. The Journal of the American Medical Association 242:1145–1146

    Article  Google Scholar 

  • Cohen-Schotanus J. (1999). Student assessment and test rules. Medical Teacher 21:318–321

    Article  Google Scholar 

  • Eiertsen T.V., Valdermo O. (2000). Open-book assessment: a contribution to improved learning? Studies in Educational Evaluation 26:91–103

    Article  Google Scholar 

  • Feldhusen J.F. (1961). An evaluation of college students’ reactions to open-book tests. Educational and Psychological Measurement 21:637–646

    Google Scholar 

  • Feller M. (1994). Open-book testing and education for the future. Studies in Educational Evaluation 20:235–238

    Article  Google Scholar 

  • Francis J. (1982). A case for open-book tests. Educational Review 24:13–26

    Article  Google Scholar 

  • Frederiksen N. (1984). The real test bias: influences of testing on teaching and learning. American Psychologist 39:193–202

    Article  Google Scholar 

  • Hoffman, A. (1997). All Tests Should be Open-Book. [Available at: http://www.summation.net/openbook.html accessed 24 aug 2006]

  • Kalish R.A. (1958). An experimental evaluation of the open book test. Journal of Educational Psychology 49: 200–204

    Article  Google Scholar 

  • Koutselini-Ioannidou M. (1997). Testing and life-long learning: open-book and closed-book test in a university course. Studies in Educational Evaluation 23:131–139

    Article  Google Scholar 

  • Lubaway W., Brandt B. (2002). A variable structure, less resource intensive modification of problem-based learning for pharmacology instruction to health science students. Naunyn-Schmiedeberg’s Archives of Pharmacology 366:48–57

    Article  Google Scholar 

  • Mohanan, K.P. (2004). Open-Book Tests: A Response to Some Recurrent Concerns. [available at: http://courses.nus.edu.sg/course/ellkpmoh/educ/cdtl-obe.pdf; accessed 24 aug 2006]

  • Nunnally, J.C. & Bernstein, I.H. (2000). Psychometric Theory (3rd ed.). 8th print. New York: McGraw-Hill

  • O’Grady, G. (2000). Open-book Tests. [Available at: www.sdtl.nus.edu.sg/link/jul2000/practice2.htm]

  • Rasbash, J., Healy, M., Browne, W., Cameron, B. & Charlton, C. (2004). MlwiN 2.02. Multilevel application for Windows

  • Shine S., Kiravu C., Astley J. (2004). In defence of open-book engineering degree examinations. International Journal of Mechanical Engineering Education 32:197–211

    Google Scholar 

  • Snijders, T.A.B & Bosker, R.J. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London, Thousand Oaks, New Delhi: SAGE

  • Spetz N.S. (1989). No right answer. History and Social Science Teacher 24:73–75

    Google Scholar 

  • Theophilides C., Dionysiou O. (1996). The major functions of the open-book test at the university level: a factor analytic study. Studies in Educational Evaluation 22:157–170

    Article  Google Scholar 

  • Theophilides C., Koutselini M. (2000). Study behavior in the closed-book and the open-book test: a comparative analysis. Educational Research and Evaluation 6:379–393

    Article  Google Scholar 

  • van der Vleuten C.P.M., Schuwirth L.W.T. (2005). Assessing professional competence: from methods to programs. Medical Education 39:309–317

    Article  Google Scholar 

  • Zeidner M. (1990). College students’ reactions towards key facets of classroom testing. Assessment and Evaluation in Higher Education 15:151–169

    Article  Google Scholar 

Download references

Acknowledgement

We are grateful to Mrs J. Bouwkamp-Timmer for her support with the literature research and her constructive comments on this manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. Heijne-penninga.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Heijne-penninga, M., Kuks, J.B.M., Schönrock-adema, J. et al. Open-book Tests to Complement Assessment-programmes: Analysis of Open and Closed-book Tests. Adv in Health Sci Educ 13, 263–273 (2008). https://doi.org/10.1007/s10459-006-9038-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10459-006-9038-y

Keywords

Navigation