Assessment of a Brazilian public hospital’s performance for management purposes: A soft operations research case in action☆
Introduction
The hospitals attached to the Federal Universities in Brazil have long ceased to focus on the training of new healthcare professionals. Social and economic constraints have diverted the focus of these institutions from training to welfare practices. Known for their quality service, the University Hospitals assist many patients from the Brazilian National Health System (SUS). However, the University Hospitals can no longer sustain on the budget share allocated to the attached public higher education institutions (University) by the Ministry of Education. These hospitals also require funding from the Ministry of Health. Moreover, the exponential increase in the demand for health assistance provided by the SUS has further led to a readjustment of budget transfer policies to reflect the current situation. The federal government thus, created an inter-ministerial commission to define the necessary alterations in the management of University Hospitals to readjust them to the new Brazilian model of public healthcare. The commission evaluated the funding, planning, teaching, and assistance needs. The commission’s final report suggested a series of modifications for the approval and release of budget funds from the SUS to the Federal University Hospitals.
The annual transfer of funds from the Ministry of Health is now contingent upon the certification of the University Hospitals by the SUS. Prior to the award or the renewal of this certificate, the following seven broad areas must be assessed: (1) health care; (2) SUS priority policies; (3) activities aimed at improving hospital management; (4) education; (5) research; (6) technological assessment and incorporation; and (7) funding.
Recognizing that each hospital is unique, the healthcare management body did not detail specific criteria for each one of these seven broad areas. Given the particularities of each University Hospital, such as size, region of location, assisted population, and delivery of postgraduate program (strictu sensu), among others, each hospital was asked to provide details on potential actions to help them meet the general certification goals. Thus, each hospital built its own individual criteria.
However, the decisions makers (hospitals’ managers) remain unsure of the criteria that need to be assessed for the certification process due to the lack of a scientific model to define these goals. Further, this process is a medium-term program that needs periodic re-analysis to maintain certification. Thus, one of the greatest challenges is the development of an instrument that allows for the systematic assessment of university hospitals’ performance in achieving their goals. Within this context, this paper presents the use of a MultiCriteria Decision Aid (MCDA) approach to support the development of a customized model to assess performance. This model is designed to aid the managers of Brazilian Federal University Hospitals identify, operate, and measure the criteria that affect a university hospital’s joint goals with the MS/MEC to obtain or renew their certification from that Ministry.
The present article is divided into six sections. Section 2 establishes the initial benchmark and explains the theoretical basis of the research. Section 3 reviews the existing literature and available methods on performance assessment models for hospital management purposes. Section 4 explains the certification requirements for the Brazilian university hospitals. Section 5 describes the construction of the assessment model for the case study. Finally, Section 6 concludes the paper with the findings and final considerations of the research.
Section snippets
Performance assessment model for hospital management purposes
The evaluation of health services in general, and in its more complex form, hospital services, present difficulties, which have been widely discussed in the literature [1], [2], [3], [4], [5], [6].
Referring specifically to hospitals, the World Health Organization (WHO) defines them as being an important part of any health care system, responsible for providing care of low, medium, and high complexity [3]. In addition, hospitals are centers of knowledge and skill development. Consequently,
The certification process of Brazilian university hospitals within the SUS
The Brazilian National Health System (SUS) was established by the Brazilian Federal Constitution of 1988 [19] and is regulated by Law no. 8080/90 [20], with the objective to shift the inequality in the health care system and mandatorily provide public health care to all, thus prohibiting the collection of monetary amounts under any pretext. The SUS network integrates health centers and clinics, hospitals, including university hospitals, laboratories, foundations and medical research
Theoretical foundation
The MCDA process of intervention operates in a systemic and systematic way through three sequential and interactive stages: the structuring stage, the evaluation stage, and the recommendation stage [23], [24]. Fig. 1 illustrates the three stages of the MCDA and their respective steps.
The first stage (structuring) focuses on understanding the problem [26], [27], [28]. Under the constructivist assumption [29], the elaboration of a structure (a more or less formal model) shared by the actors of
Construction of the assessment model
Many contexts involve more than one actor, each with distinct: interests; values; and preference system. In these contexts the conflict with respect to important objectives is frequent. Whatever the source of the distinct value systems, they will usually affect the evolution of the decision process [46]. The present case study deals with a situation similar where the general manager does not want to dictate decisions, instead she wants to have a participative contribution of all operational
Final considerations
The present study describes the use of the MultiCriteria Decision Aid (MCDA) as an intervention instrument in a case study performed at a University Hospital of the Brazilian federal education system. The study aimed to develop a customized performance assessment model to help the managers of that hospital identify, operate, and measure the criteria to attain agreed upon goals as required to obtain certification from the MS/SUS.
The study was conducted in a systemic and systematic way in three
Acknowledgments
We thank http://Editage.com for their translation service, our anonymous reviewers for their important contributions to the work, and the Brazilian Bureau of Research (CNPq) for funding this research.
References (66)
Value: Operations research and the new health care paradigm
Oper. Res. Health Care
(2012)- et al.
Multi-criteria decision analysis (MCDA) in health care: A bibliometric analysis
Oper. Res. Health Care
(2013) Robustness in operational research and decision aiding: A multi-faceted issue
European J. Oper. Res.
(2010)Decision science or decision-aid science?
European J. Oper. Res.
(1993)- et al.
MCDA: a constructivist approach to the management of human resources at a governmental agency
Int. Trans. Oper. Res.
(2000) - et al.
The multicriteria analysis for construction of bench markers to support the Clinical Engineering in the Healthcare Technology Management
European J. Oper. Res.
(2010) - et al.
Decision support systems in action: integrated application in multicriteria decision aid process
European J. Oper. Res.
(1999) - et al.
In search of a valid view of model validation for operations research
European J. Oper. Res.
(1996) - et al.
Modelo legitimization in operational research
European J. Oper. Res.
(1996) A foundational concept for validation in operational research
European J. Oper. Res.
(1993)
The facets of the modelling and validation process in operational research
European J. Oper. Res.
Revisiting the issue of model validation in OR: An Epistemological view
European J. Oper. Res.
A career choice problem: An example of how to use MACBETH to build a quantitative value model based on qualitative value judgments
European J. Oper. Res.
Use of OR by government to inform health policy in England: Examples and reflections
Oper. Res. Health Care
A review of organizational performance assessment in health Care
Health Serv. Manag. Res.
Health systems performance assessment in low-income countries: learning from international experiences
Glob. Health
A performance assessment framework for hospitals: the WHO regional office for Europe PATH Project
Int. J. Qual. Health Care
Performance measurement system for healthcare process
Int. J. Product. Perform. Manag.
Prioritizing health care interventions: A multicriteria resource allocation model to inform the choice of community care programmes
Evaluation of Brazil’s public health surveillance system within the context of the International Health Regulations (2005)
Rev. Panam. Salud Publica
Hospital financial condition and operational decisions related to the quality of hospital care
Med. Care Res. Rev.
How can clinical practice guidelines be adapted to facilitate shared decision making? A qualitative key-informant study
BMJ Qual. Saf.
Priority-setting in healthcare: A framework for reasonable clinical judgements
J. Med. Ethics
Measurement theory
A performance measurement framework in portfolio management: A constructivist case
Manag. Decis.
A performance measurement view of IT project management
Int. J. Product. Perform. Manag.
Constituição da República Federativa do Brasil
Cited by (16)
Using MCDA-C to assess the organizational performance of industries operating at Brazilian maritime port terminals
2019, Operations Research PerspectivesCitation Excerpt :MCDA-C consists of a set of methods to help people and organizations make decisions, clarify the problem, and evaluate alternatives through multiple criteria, which are, in most cases, conflicting. Multicriteria decision aid methods typically aim to list (subjective) preferences among the alternatives being evaluated, under the influence of many criteria in the decision-making process [21]. The activities of MCDA-C are subdivided into three stages: structuring, evaluation, and recommendation development [22].
Multivariate analysis techniques applied for the performance measurement of Federal University Hospitals of Brazil
2018, Computers and Industrial EngineeringCitation Excerpt :This scenario corroborates with the fact that the efficiency inherent in each federal university hospital, accompanied by the achievement of productivity goals based on, health care areas, SUS priority policies, activities to improve hospital management, education, research, technological evaluation and integration, as well as financing, represents the agreement signed between HUFs and the Ministry of Health. This was in order to obtain certification, guaranteeing the transfer of funds to these institutions (Lins, Lobo, Silva, Fiszman, & Ribeiro, 2007; Longaray, Ensslin, Ensslin, & da Rosa, 2015; Ozcan et al., 2010). However, the fourth and first groups, composed of 20 and 45 medical specialties, respectively, presented basically the same values for the output variable, medical residency, which were, respectively, 3.86 and 3.67 units, by area, with Group III exceeding them, on average, in only one unit (4.52).
Developing criteria for performance assessment in municipal solid waste management
2018, Journal of Cleaner ProductionCitation Excerpt :The mathematical relationships can be obtained through support given by M-MACBETH software, which implements the method computationally. According to Longaray et al. (2015), in so doing it, the judgmental analyzes of the decision maker are made. When converting the ordinal scales, descriptors, into cardinal, the M-MACBETH allows the setting of the intervals: Neutral (0) and Good (100) levels, in addition to normalizing the scale.
Structuring problems for Multi-Criteria Decision Analysis in practice: A literature review of method combinations
2017, European Journal of Operational ResearchCitation Excerpt :The most common reason for using CMs/GMs was to facilitate the development of a comprehensive set of indicators (Adrianto, Matsuda, & Sakuma, 2005; Bana e Costa et al., 2006; Ferreira et al., 2012; Marafon, Ensslin, Lacerda, & Ensslin, 2015; Mendoza & Prabhu, 2003; Myllyviita et al., 2014). Other motivations were to structure indicators (Wolfslehner & Vacik, 2011), to assess their significance by calculating their centrality and domain (Adrianto et al., 2005), to help the decision-makers to understand, delineate and organise the problem (Longaray, Ensslin, Ensslin, & da Rosa, 2015), to elicit and structure the stakeholders' knowledge of the problem situation and to stimulate new causal thinking (Franco & Lord, 2011). The intensity of the participants’ involvement varied from very high to none.
Using constructivist multi-criteria decision aid to evaluate the workforce sizing in public organisations: the case of a Brazilian university
2023, International Journal of Operational Research
- ☆
This paper is an expanded version of the article originally published in Portuguese by Longaray and Ensslin (2014).