Skip to main content

Advertisement

SpringerLink
Log in

Examining benchmark indicator systems for the evaluation of higher education institutions

  • Published:
Higher Education Aims and scope Submit manuscript

Abstract

Higher Education Institutions are undergoing important changes involving the development of new roles and missions, with implications for their structure. Governments and institutions are implementing strategies to ensure the proper performance of universities and several studies have investigated evaluation of universities through the development and use of indicator systems. In this paper, we review some of the systems applied to the OECD countries, with special attention to Spain. We demonstrate the difficulty involved in establishing classification criteria for existing indicators, on which there is currently no consensus.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • ABET. (2006). Accreditation policy and procedure manual. Effective for evaluations during the 2007–2008 accreditation cycle. http://www.abet.org/.

  • Aghion, P., Dewatripont, M., Hoxby, C., Mas-Colell, A., & Sapir, A. (2007). Why reform Europe’s universities? Bruegel Policy Brief, 4, 1–8.

    Google Scholar 

  • ANECA. (2008). Manual de procedimiento para la evaluación de las solicitudes de implantación de títulos oficiales de grado y master. http://www.aneca.es/.

  • AQU. (2007). El Sistema Universitari Públic Català 2000–2005: Una Perspectiva des de l’avaluació d’AQU Catalunya. Barcelona: AQU.

    Google Scholar 

  • AUCC. (2008). The Directory of Canadian Universities. http://www.aucc.ca/publications/auccpubs/directories/dcu/dcu_e.html.

  • Blank, R. K. (1993). Developing a system of education indicators: Selecting, implementing and reporting indicators. Educational Evaluation and Policy Analysis, 15(1), 65–80.

    Google Scholar 

  • Bonaccorsi, A., & Daraio, C. (2007). Universities as strategic knowledge creators: Some preliminary evidence. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation (pp. 31–81). Northampton: Edward Elgar.

    Google Scholar 

  • Bricall, J. M. (2000). Universidad 2mil. Madrid: CRUE.

    Google Scholar 

  • Bueno Campos, E. (2007). La tercera misión de la universidad. El reto de la transferencia del conocimiento. Revista madri+d, 41, marzo-abril.

  • Castro-Martínez, E., Cortes-Herreros, A. M., Nicolás-Gelonch, M., & Costa-Leja, C. (2005). Una aproximación al análisis de impacto de las universidades en su entorno a través de un estudio de las actividades de las OTRI universitarias españolas. Paper presented at XI Seminario Latino Iberoamericano de Gestión Tecnológica, ALTEC 2005, Salvador de Bahía (Brasil).

  • Cave, M., Hanney, S., Henkel, M., & Kogan, M. (1988, 1997). The use of performance indicators in higher education (1st & 3rd ed.). London and Bristol: Jessica Kingsley Publishers.

  • CESC. (2005). Education indicators in Canada, 2005. http://www.cesc.ca/pceipE.html.

  • Chacón Moscoso, S., Pérez Gil, J. A., & Holgado Tello, F. P. (1999). Indicadores de calidad en la universidad de Sevilla. Análisis de contenido. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 385–400). Madrid: MEC.

    Google Scholar 

  • CIHE. (2007). Standards for accreditation. Commission on institutions of higher education. New England Association of Schools and Colleges. http://www.neasc.org/cihe/standards_for_accreditation_2005.pdf.

  • Clark, B. R. (1997). Creando universidades emprendedoras en Europa. Revista Valenciana d’Estudis Autonomics, 21, 373–392.

    Google Scholar 

  • Clark, B. R. (1998). The entrepreneurial university demand and response. Tertiary Education Management, 4(1), 5–15.

    Google Scholar 

  • Clark, D., Lotto, L. S., & Astuto, T. A. (1984). Effective schools and school improvement: A comparative analysis two lines inquiry. Educational Administration Quarterly, 20(3), 41–68.

    Article  Google Scholar 

  • CNE. (2003). Livre des références. Paris: Comité National d’Évaluation.

    Google Scholar 

  • Commission of the European Communities. (2006). Communication from the commission—delivering on the modernisation agenda for universities: Education, research and innovation. Brussels: Commission of the European Communities.

    Google Scholar 

  • Consejo de Coordinación Universitaria. (2002). Catálogo de Indicadores del Sistema Universitario Público Español. Madrid: MEC.

    Google Scholar 

  • Consejo de Universidades. (1999). Indicadores en la Universidad: Información y decisiones. Madrid: MEC.

    Google Scholar 

  • Cuenin, S. (1987). International study of the development of performance indicator in higher education. Paper from to OECD, IMHE Project, Special Topic Workshop.

  • Davies, J. L. (2001). The emergence of entrepreneurial cultures in European universities. Higher Education Management, 13(2), 25–43.

    Google Scholar 

  • De Miguel, M. (1989). Modelos de investigación sobre organizaciones educativas. Revista de Investigación Educativa, 7(13), 21–56.

    Google Scholar 

  • De Miguel, M. (2007). Acreditación de educación superior. Madrid: CIS.

    Google Scholar 

  • De Miguel, M., Madrid, V., Noriega, J., & Rodríguez, B. (1994). Evaluación para la calidad de los institutos de educación secundaria. Madrid: Escuela Española.

    Google Scholar 

  • De Pablos Escobar, L., & Gil Izquierdo, M. (2004). Algunos indicadores de eficiencia, eficacia y calida del sistema de financiación de la universidad. In J. Grao Rodríguez (Ed.), Economía de la educación. San Sebastián: Actas de las XIII jornadas de la AEDE.

    Google Scholar 

  • Douglas Williams, J. (1995). The challenge of developing new educational indicators. Educational Evaluation and Policy Analysis, 17(1), 113–131.

    Article  Google Scholar 

  • Eaton, J. S. (2006). An overview of U.S. accreditation. Santa Barbara: CHEA.

    Google Scholar 

  • El-Khawas, E., Depieto-Jurand, R., & Hola-Nielsen, L. (1998). Quality assurance in higher education: Recent progress; challenges ahead. Paris: World Bank.

    Google Scholar 

  • ENQA. (2007). Standards and guidelines for quality assurance in the European Higher Education Area. ENQA. http://www.enqa.eu/files/ESG_v03.pdf.

  • Etzkowitz, H., & Leydesdorff, L. (1997). University and the global knowledge economy. A triple helix of university- industry- government relations. London: Pinter Publishers.

    Google Scholar 

  • Etzkowitz, H., Webster, A., Gebhardt, C., & Catisano, B. R. (2000). The future of the university and the University of the Future: Evolution of ivory tower to entrepreneurial paradigm. Research Policy, 29, 313–330.

    Article  Google Scholar 

  • European Commission. (2003). Third European report on science and technology indicators, 2003. Brussels: European Commission.

    Google Scholar 

  • European Commission. (2005). European universities: Enhancing Europe’s research base. Brussels: European Commission.

    Google Scholar 

  • Eurostat. (2007). Total intramural RandD expenditure (GERD) by source of funds. Luxembourg: Eurostat.

    Google Scholar 

  • Fernández, J. M. (1999). Manual de Políticas y Legislación Educativa. Madrid: Síntesis Educación.

    Google Scholar 

  • García-Aracil, A. (2007). Expansion and reorganization in the Spanish higher education system. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation (pp. 376–401). Northampton: Edward Elgar.

    Google Scholar 

  • García-Aracil, A., & Daraio, C. (2009). The efficiency of Spanish Public Universities: An investigation of economies of scale and scope, and trade-offs in teaching versus research. Economics of education review (in press).

  • García-Aracil, A., Gutiérrez-Gracia, A., & Pérez-Marín, M. (2006). Analysis of the evaluation process of the research performance: An empirical case. Scientometrics, 67(2), 213–230.

    Article  Google Scholar 

  • García-Aracil, A., & Villarreal, E. (2009). Some indicators to measure regional impact of entrepreneurial universities. Management Science (in press).

  • García Ramos, J. M. (1989). Extensión de la evaluación. In R. Pérez Juste & J. M. García Ramos (Eds.), Diagnóstico, evaluación y toma de decisiones (pp. 315–385). Madrid: Rialp.

    Google Scholar 

  • Geuna, A. (1999). An evolutionary account of European universities. In A. Geuna (Ed.), The economics of knowledge production. Funding and the structure of university research (pp. 35–60). Northampton: Edward Elgar.

    Google Scholar 

  • Gibbons, M. (1999). Science’s new social contract with society. Nature, 402, 11–18.

    Article  Google Scholar 

  • Giménez García, V., & Martínez Parra, J. L. (2006). Cost efficiency in the university. A departmental evaluation model. Economics of Education Review, 25(5), 543–553.

    Article  Google Scholar 

  • González Fernández, N., Osorio Sierra, J. M., & Salvador Blanco, L. (1999). Necesidad de indicadores para la evaluación del practicum en las titulaciones de ciencias de la educación. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 367–374). Madrid: MEC.

    Google Scholar 

  • Guerra, C., Lecue, M., Rueda, E. M., & Vázquez, P. (1999). Un sistema de indicadores para analizar el perfil de los departamentos de una universidad generalista. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 157–186). Madrid: MEC.

    Google Scholar 

  • Gulbrandsen, M., & Slipersaeter, S. (2007). The third mission and the entrepreneurial university model. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation (pp. 112–143). Northampton: Edward Elgar.

    Google Scholar 

  • HEFCE. (2008). Strategic plan 2006–2011. http://www.hefce.ac.uk/pubs/hefce/2008/08_15/08_15.pdf.

  • Hernández Armenteros, J. (2003). Universidades e Indicadores de Gestión: El documento de la Conferencia de Rectores de las Universidades Españolas. Año 2000. Paper presented at Jornadas ‘Indicadores sobre la calidad universitaria’. Universidad Carlos III, Madrid (España).

  • HLC. (2003). Institutional accreditation: An overview. Chicago: Commission of the North Central Association of Colleges and Schools.

    Google Scholar 

  • INE. (2007). Estadística sobre desarrollo de I+D. 2007. Madrid: INE.

    Google Scholar 

  • Joumady, O., & Ris, C. (2005). Determining the relative efficiency of European higher education institutions using DEA. Maastricht: ROA Maastricht University.

    Google Scholar 

  • Keller, G. (1999). The New importance of strategic management at universities. Universitat: Estratègies per avançar. Barcelona: UPC.

  • Laredo, P. (2007). Revisiting the third mission of universities: Toward a renewed categorization of university activities? Higher Education Policy, 20(4), 441–456.

    Article  Google Scholar 

  • Lázaro, A. (1992). La formalización de los indicadores de evaluación. Bordón, 43(4), 477–494.

    Google Scholar 

  • Lepori, B., Benninghoff, M., Jongbloed, B., Salerno, C., & Slipersaeter, S. (2007). Changing models and patterns of higher education funding: Some empirical evidence. In A. Bonaccorsi & C. Daraio (Eds.), Universities and strategic knowledge creation (pp. 85–111). Northampton: Edward Elgar.

    Google Scholar 

  • Leydesdorff, L. (2006). The knowledge-based economy: Modeled, measured, simulated. Boca Raton: Universal Publisher.

    Google Scholar 

  • LOU. (2001). BOE n. 307 de 24/12/2001.

  • LOU. (2007). BOE n. 89 de 13/04/2007.

  • Manley, K. (2002). The systems approach to innovation studies. Australian Journal of Information Systems, 9(2), 94–102.

    Google Scholar 

  • Martín, C. J. (2000). La misión de la universidad en el S.XXI: ¿Torre de marfil, parque de diversión, oficina comercial o pensar lo impensable? Estudios del Hombre, 12, 43–59.

    Google Scholar 

  • Martin, B., & Etzkowitz, H. (2000). The origin and evolution of the university species. VEST, 13, 9–34.

    Google Scholar 

  • MEC. (2007). Borrador del catálogo de indicadores del sistema universitario público español. Madrid: MEC.

    Google Scholar 

  • Miguel Díaz, M. (1999). La evaluación de la enseñanza. Propuesta de indicadores para las titulaciones. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 413–430). Madrid: MEC.

    Google Scholar 

  • Molas-Gallart, J. (2002). Measuring third stream activities. United Kingdom: University of Sussex.

    Google Scholar 

  • Molas-Gallart, J., & Castro-Martínez, E. (2006). Ambiguity and conflict in the development of ‘Third Mission’ indicators. Paper presented at the 9th International Conference on Science and Technology Indicators, Lovaina.

  • Molas-Gallart, J., & Ordóñez, S. (2006). Las escuelas de arquitectura frente a la Tercera Misión. Paper presented at Segundas Jornadas sobre Investigación en Arquitectura y Urbanismo. UPC, Barcelona.

  • Mora, J. G. (1991). Evaluación y rendimiento institucional. Quaderns de Treball, 199.

  • Mora, J. G. (1999). Indicadores y decisiones en las universidades. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 19–30). Madrid: MEC.

    Google Scholar 

  • Murnane, R. J. (1987). Improving education indicators and economic indicators: The same problems? Educational Evaluation and Policy Analysis, 9(2), 101–116.

    Google Scholar 

  • Oakes, J. (1989). What education indicators? The case for assessing the school context. Educational Evaluation and Policy Analysis, 11(2), 182–199.

    Google Scholar 

  • OECD. (1990). Proposed standard method of compiling and interpreting technology balance of payments data. TBP manual. Paris: OECD.

    Google Scholar 

  • OECD. (1994). The measurement of scientific and technological activities. Using patent data as science and technology indicators. Patent manual. Paris: OECD.

    Google Scholar 

  • OECD. (1999). University research in transition. Paris: OECD.

    Google Scholar 

  • OECD. (2002). Frascati manual. Paris: OECD.

    Book  Google Scholar 

  • OECD. (2005). Manual de Oslo. Guía para la recogida e interpretación de datos sobre innovación. Paris: OECD.

    Google Scholar 

  • OECD. (2007). Education at a glance. OECD indicators. Paris: OECD.

    Google Scholar 

  • OEI. (1998). Declaración Mundial sobre la Educación Superior en el Siglo XXI: Visión y Acción. París: UNESCO.

    Google Scholar 

  • Pla, A., & Villarreal, E. (2001). Measuring the internal efficiency of universities: Social demand, diversity and rising costs. Paper presented at the XXIII Annual EAIR Forum. University of Porto, Porto.

  • ProTon. (2007). The ProTon Europe 2005. Annual survey report. Valencia: INGENIO, CSIC-UPV.

    Google Scholar 

  • QAA. (2006). Handbook for institutional audit: England and Northern Ireland, 2006. Mansfield: AQQ.

    Google Scholar 

  • Rodríguez Espinar, S. (1999). Información cualitativa y cuantitativa en el Plan Nacional de Evaluación. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 401–412). Madrid: MEC.

    Google Scholar 

  • Rodríguez Vargas, J. J. (2005). La Nueva Fase de Desarrollo Económico y Social del Capitalismo Mundial. http://www.eumed.net/tesis/jjrv/.

  • SACS. (2008). Principles of accreditation foundations for quality enhancement. http://www.sacscoc.org/pdf/2008PrinciplesofAccreditation.pdf.

  • Scott, P. (1998). Fisuras en la Torre de marfil. El correo de la Unesco, Septiembre, 18–20.

  • Stassen, M. L. A., Doherty, D., & Poe, M. (2001). Program-based review and assessment. US: University of Massachusetts.

    Google Scholar 

  • Tricio, A., Bol, A., Vitoria, R., Quindos, M. P., & Serna, M. (1999). Una experiencia departamental en evaluación de la calidad y adaptación de los indicadores para investigación, enseñanza y gestión. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 231–250). Madrid: MEC.

    Google Scholar 

  • UNESCO. (2003). system-level and strategic indicators for monitoring higher education in the twenty-first century. Bucharest: UNESCO.

    Google Scholar 

  • UNESCO. (2004). Studies on higher education. Indicators for institutional and programme accreditation in higher tertiary education. Bucharest: UNESCO.

    Google Scholar 

  • Villarreal, E. (1999). La utilización de indicadores de rendimiento en la financiación de la educación superior. In Consejo de Universidades (Ed.), Indicadores en la Universidad: Información y decisiones (pp. 65–80). Madrid: MEC.

    Google Scholar 

  • Westerheijden, D. F. (1999). Innovations indicators in science and technology: Comments from a higher education point of view. Scientometrics, 45(3), 445–453.

    Article  Google Scholar 

  • Wimpelberg, R., Teddlie, C., & Stringfield, S. (1989). Sensitivity to context: The past and the future of effective school research. Educational Administration Quarterly, 25(1), 82–127.

    Article  Google Scholar 

Download references

Acknowledgments

This paper was supported by the research project ref. ECO2008-02553/ECON, named “The University Missions and Their Complementarity. New Methods of Evaluating Efficiency”, funds provided by the National R&D Programme of the Spanish Ministry of Science and Innovation. This organization is not responsible for the views expressed.

Author information

Authors and Affiliations

  1. Institute for Innovation and Knowledge Management, Valencia, Spain

    Adela García-Aracil & Davinia Palomares-Montero

Authors
  1. Adela García-Aracil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Davinia Palomares-Montero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adela García-Aracil.

Rights and permissions

Reprints and permissions

About this article

Cite this article

García-Aracil, A., Palomares-Montero, D. Examining benchmark indicator systems for the evaluation of higher education institutions. High Educ 60, 217–234 (2010). https://doi.org/10.1007/s10734-009-9296-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10734-009-9296-8

Keywords

Search

Navigation