Assessment of a Brazilian public hospital’s performance for management purposes: A soft operations research case in action

https://doi.org/10.1016/j.orhc.2015.05.001Get rights and content

Abstract

Brazil’s public health management system has undergone substantial restructuring over the past decade. One of the consequences of this restructuring is the establishment of a new relationship between the Ministry of Health (MS) and the Ministry of Education (MEC) regarding the funding of Federal University Hospitals. The Brazilian Ministry of Health has implemented a certification process for the University Hospitals that requires meeting a set of financial, managerial, and educational processes related goals, and adoption of an assistance model. This article uses the MultiCriteria Decision Aid (MCDA) approach to support the development of a model that can help the university hospitals’ managers assess their institutional performance to meet their joint goals with the MS/MEC and obtain the certification. This case study was conducted at a hospital within the Brazilian federal education network.

Introduction

The hospitals attached to the Federal Universities in Brazil have long ceased to focus on the training of new healthcare professionals. Social and economic constraints have diverted the focus of these institutions from training to welfare practices. Known for their quality service, the University Hospitals assist many patients from the Brazilian National Health System (SUS). However, the University Hospitals can no longer sustain on the budget share allocated to the attached public higher education institutions (University) by the Ministry of Education. These hospitals also require funding from the Ministry of Health. Moreover, the exponential increase in the demand for health assistance provided by the SUS has further led to a readjustment of budget transfer policies to reflect the current situation. The federal government thus, created an inter-ministerial commission to define the necessary alterations in the management of University Hospitals to readjust them to the new Brazilian model of public healthcare. The commission evaluated the funding, planning, teaching, and assistance needs. The commission’s final report suggested a series of modifications for the approval and release of budget funds from the SUS to the Federal University Hospitals.

The annual transfer of funds from the Ministry of Health is now contingent upon the certification of the University Hospitals by the SUS. Prior to the award or the renewal of this certificate, the following seven broad areas must be assessed: (1) health care; (2) SUS priority policies; (3) activities aimed at improving hospital management; (4) education; (5) research; (6) technological assessment and incorporation; and (7) funding.

Recognizing that each hospital is unique, the healthcare management body did not detail specific criteria for each one of these seven broad areas. Given the particularities of each University Hospital, such as size, region of location, assisted population, and delivery of postgraduate program (strictu sensu), among others, each hospital was asked to provide details on potential actions to help them meet the general certification goals. Thus, each hospital built its own individual criteria.

However, the decisions makers (hospitals’ managers) remain unsure of the criteria that need to be assessed for the certification process due to the lack of a scientific model to define these goals. Further, this process is a medium-term program that needs periodic re-analysis to maintain certification. Thus, one of the greatest challenges is the development of an instrument that allows for the systematic assessment of university hospitals’ performance in achieving their goals. Within this context, this paper presents the use of a MultiCriteria Decision Aid (MCDA) approach to support the development of a customized model to assess performance. This model is designed to aid the managers of Brazilian Federal University Hospitals identify, operate, and measure the criteria that affect a university hospital’s joint goals with the MS/MEC to obtain or renew their certification from that Ministry.

The present article is divided into six sections. Section  2 establishes the initial benchmark and explains the theoretical basis of the research. Section  3 reviews the existing literature and available methods on performance assessment models for hospital management purposes. Section  4 explains the certification requirements for the Brazilian university hospitals. Section  5 describes the construction of the assessment model for the case study. Finally, Section  6 concludes the paper with the findings and final considerations of the research.

Section snippets

Performance assessment model for hospital management purposes

The evaluation of health services in general, and in its more complex form, hospital services, present difficulties, which have been widely discussed in the literature  [1], [2], [3], [4], [5], [6].

Referring specifically to hospitals, the World Health Organization (WHO) defines them as being an important part of any health care system, responsible for providing care of low, medium, and high complexity  [3]. In addition, hospitals are centers of knowledge and skill development. Consequently,

The certification process of Brazilian university hospitals within the SUS

The Brazilian National Health System (SUS) was established by the Brazilian Federal Constitution of 1988  [19] and is regulated by Law no. 8080/90  [20], with the objective to shift the inequality in the health care system and mandatorily provide public health care to all, thus prohibiting the collection of monetary amounts under any pretext. The SUS network integrates health centers and clinics, hospitals, including university hospitals, laboratories, foundations and medical research

Theoretical foundation

The MCDA process of intervention operates in a systemic and systematic way through three sequential and interactive stages: the structuring stage, the evaluation stage, and the recommendation stage  [23], [24]. Fig. 1 illustrates the three stages of the MCDA and their respective steps.

The first stage (structuring) focuses on understanding the problem  [26], [27], [28]. Under the constructivist assumption  [29], the elaboration of a structure (a more or less formal model) shared by the actors of

Construction of the assessment model

Many contexts involve more than one actor, each with distinct: interests; values; and preference system. In these contexts the conflict with respect to important objectives is frequent. Whatever the source of the distinct value systems, they will usually affect the evolution of the decision process  [46]. The present case study deals with a situation similar where the general manager does not want to dictate decisions, instead she wants to have a participative contribution of all operational

Final considerations

The present study describes the use of the MultiCriteria Decision Aid (MCDA) as an intervention instrument in a case study performed at a University Hospital of the Brazilian federal education system. The study aimed to develop a customized performance assessment model to help the managers of that hospital identify, operate, and measure the criteria to attain agreed upon goals as required to obtain certification from the MS/SUS.

The study was conducted in a systemic and systematic way in three

Acknowledgments

We thank http://Editage.com for their translation service, our anonymous reviewers for their important contributions to the work, and the Brazilian Bureau of Research (CNPq) for funding this research.

References (66)

  • M. Oral et al.

    The facets of the modelling and validation process in operational research

    European J. Oper. Res.

    (1993)
  • R. Déry et al.

    Revisiting the issue of model validation in OR: An Epistemological view

    European J. Oper. Res.

    (1993)
  • C.A. Bana e Costa et al.

    A career choice problem: An example of how to use MACBETH to build a quantitative value model based on qualitative value judgments

    European J. Oper. Res.

    (2004)
  • P. Bennet et al.

    Use of OR by government to inform health policy in England: Examples and reflections

    Oper. Res. Health Care

    (2012)
  • World Health Organization. A review of determinants of hospital performance: Report of the WHO Hospital Advisory Group...
  • S.G. Leggat et al.

    A review of organizational performance assessment in health Care

    Health Serv. Manag. Res.

    (1998)
  • World Health Organization. Measuring Hospital Performance to improve the quality of care in Europe: A need for...
  • C.K. Tashobya

    Health systems performance assessment in low-income countries: learning from international experiences

    Glob. Health

    (2014)
  • J. Veillard

    A performance assessment framework for hospitals: the WHO regional office for Europe PATH Project

    Int. J. Qual. Health Care

    (2005)
  • S. Purbey et al.

    Performance measurement system for healthcare process

    Int. J. Product. Perform. Manag.

    (2007)
  • M. Oliveira et al.

    Prioritizing health care interventions: A multicriteria resource allocation model to inform the choice of community care programmes

  • M.G. Teixeira et al.

    Evaluation of Brazil’s public health surveillance system within the context of the International Health Regulations (2005)

    Rev. Panam. Salud Publica

    (2012)
  • G.J. Bazzoli et al.

    Hospital financial condition and operational decisions related to the quality of hospital care

    Med. Care Res. Rev.

    (2007)
  • T. Van Der Weijden et al.

    How can clinical practice guidelines be adapted to facilitate shared decision making? A qualitative key-informant study

    BMJ Qual. Saf.

    (2013)
  • K. Bærøe

    Priority-setting in healthcare: A framework for reasonable clinical judgements

    J. Med. Ethics

    (2009)
  • U.M. Waldmann, M. Gulich, et al. A complex process: Decision-making in general practice: 117 Structured case analyses,...
  • F. Roberts

    Measurement theory

  • R.T. Lacerda et al.

    A performance measurement framework in portfolio management: A constructivist case

    Manag. Decis.

    (2011)
  • R.T. Lacerda et al.

    A performance measurement view of IT project management

    Int. J. Product. Perform. Manag.

    (2011)
  • Constituição da República Federativa do Brasil

    (1988)
  • Brasil. Lei no. 8.080, de 19 de setembro de 1990, Dispõe sobre as condições para a promoção, proteção e recuperação da...
  • Brasil. Ministério da Saúde, Resolução no. 562, de 12 de maio de 2003, Dispõe sobre a criação da comissão...
  • Brasil. Ministério da Saúde, Portaria no. 2.400 de 02 de outubro de 2007, Estabelece os requisitos para a certificação...
  • Cited by (16)

    • Using MCDA-C to assess the organizational performance of industries operating at Brazilian maritime port terminals

      2019, Operations Research Perspectives
      Citation Excerpt :

      MCDA-C consists of a set of methods to help people and organizations make decisions, clarify the problem, and evaluate alternatives through multiple criteria, which are, in most cases, conflicting. Multicriteria decision aid methods typically aim to list (subjective) preferences among the alternatives being evaluated, under the influence of many criteria in the decision-making process [21]. The activities of MCDA-C are subdivided into three stages: structuring, evaluation, and recommendation development [22].

    • Multivariate analysis techniques applied for the performance measurement of Federal University Hospitals of Brazil

      2018, Computers and Industrial Engineering
      Citation Excerpt :

      This scenario corroborates with the fact that the efficiency inherent in each federal university hospital, accompanied by the achievement of productivity goals based on, health care areas, SUS priority policies, activities to improve hospital management, education, research, technological evaluation and integration, as well as financing, represents the agreement signed between HUFs and the Ministry of Health. This was in order to obtain certification, guaranteeing the transfer of funds to these institutions (Lins, Lobo, Silva, Fiszman, & Ribeiro, 2007; Longaray, Ensslin, Ensslin, & da Rosa, 2015; Ozcan et al., 2010). However, the fourth and first groups, composed of 20 and 45 medical specialties, respectively, presented basically the same values for the output variable, medical residency, which were, respectively, 3.86 and 3.67 units, by area, with Group III exceeding them, on average, in only one unit (4.52).

    • Developing criteria for performance assessment in municipal solid waste management

      2018, Journal of Cleaner Production
      Citation Excerpt :

      The mathematical relationships can be obtained through support given by M-MACBETH software, which implements the method computationally. According to Longaray et al. (2015), in so doing it, the judgmental analyzes of the decision maker are made. When converting the ordinal scales, descriptors, into cardinal, the M-MACBETH allows the setting of the intervals: Neutral (0) and Good (100) levels, in addition to normalizing the scale.

    • Structuring problems for Multi-Criteria Decision Analysis in practice: A literature review of method combinations

      2017, European Journal of Operational Research
      Citation Excerpt :

      The most common reason for using CMs/GMs was to facilitate the development of a comprehensive set of indicators (Adrianto, Matsuda, & Sakuma, 2005; Bana e Costa et al., 2006; Ferreira et al., 2012; Marafon, Ensslin, Lacerda, & Ensslin, 2015; Mendoza & Prabhu, 2003; Myllyviita et al., 2014). Other motivations were to structure indicators (Wolfslehner & Vacik, 2011), to assess their significance by calculating their centrality and domain (Adrianto et al., 2005), to help the decision-makers to understand, delineate and organise the problem (Longaray, Ensslin, Ensslin, & da Rosa, 2015), to elicit and structure the stakeholders' knowledge of the problem situation and to stimulate new causal thinking (Franco & Lord, 2011). The intensity of the participants’ involvement varied from very high to none.

    View all citing articles on Scopus

    This paper is an expanded version of the article originally published in Portuguese by Longaray and Ensslin (2014).

    View full text