Organization of Research and Innovation: a Comparative Study of Public Agricultural Research Institutions

 

Adriana Bin¹, Cecilia Gianoni², Paule J. V Mendes³, Carolina Rio⁴, Sergio L.M. Salles-Filho⁵, Luiza M. Capanema⁶

¹Assistant professor, School of Applied Sciences, University of Campinas, R. Pedro Zacarias 1300, Limeira, SP Brazil - 13484-350. Phone: 55 19 3701-6685. E-mail: adriana.bin@fca.unicamp.br

²PhD student, Department of Science and Technology Policy, University of Campinas, POBox 6152, Campinas, SP, Brazil - 13083-870. Phone: 55 19 3521 4555. Email: cecilia.gianoni@gmail.com

³Personnel Management Department, Brazilian Agricultural Research Corporation (Embrapa), Parque Estação Biológica - PqEB s/n°, Brasilia, DF, Brazil - 70770-901. Phone: 55 61 3448-4443. E-mail: paule.mendes@embrapa.br

PhD student, Department of Science and Technology Policy, University of Campinas, POBox 6152, Campinas, SP, Brazil - 13083-870. Phone: 55 19 3521 4555. E-mail: ⁴geo.rio.carol@gmail.com, ⁵salles@ige.unicamp.br, ⁶luizamcb@gmail.com

---

Abstract

This paper discusses the institutional reorganization that has culminated in new designs for the management models of Public Research Institutes (PRIs), comparing the experiences of four PRIs in the agricultural sector to identify common elements and differences in processes, flows and policies. The focus is to improve research and innovation management, developing new approaches and tools.

The four PRIs analyzed are the US Department of Agriculture's Agricultural Research Service (ARS), the Research Branch of Agriculture & Agri-Food Canada (AAFC), the Brazilian Agricultural Research Corporation (Embrapa), and Uruguay's National Agricultural Research Institute (INIA).

The study shows that management processes have become more sophisticated and above all oriented to the creation of products, processes and services that are more likely to be used. This movement indicates a new positioning by PRIs in the agricultural innovation systems of their countries, reorienting their relationships with other actors of particular importance to knowledge production.

Keywords: public research institutions; planning and management of st&i; innovation; agriculture.

---

 

Introduction

The organizational and management models adopted by research institutions have changed significantly in the past three decades. While the changes implemented in the 1980s and 1990s reflected a need for institutions to be more flexible and autonomous, the imperatives of the last decade have centered on the importance of equipping research institutions to appropriate and use the knowledge and the technologies created within innovation systems more effectively. Public research institutions (PRIs) in the agricultural field are no exception. The main consequences of the changes implemented by these PRIs have been the adoption of more sophisticated management models, especially in terms of the configuration and systematization of practices in prospecting, programming, monitoring and evaluating their research activities, as well as the incorporation into such practices of the perspective of promoting innovation in agriculture.

This paper sets out to contextualize the emergence of this new configuration of organizational and management models in the agricultural research activities of PRIs and discuss some cases that evidence such changes. The paper is divided into three sections. The first section presents an overview of the role of PRIs in the sphere of innovation systems, discussing the motivations and key consequences of the reorganization experienced by these institutions from the 1980s on, and highlighting some specificities of the agriculture sector. The second section focuses on agricultural PRIs and presents case studies of four institutions: the US Department of Agriculture's Agricultural Research Service (ARS), the Research Branch of Agriculture & Agri-Food Canada (AAFC), the Brazilian Agricultural Research Corporation (Embrapa), and Uruguay's National Agricultural Research Institute (INIA). All four are national agricultural research institutions of outstanding significance in their respective countries, where agriculture and agribusiness are important economic activities. The case studies focus on the models in place at these institutions for organizing and managing their research, and also on the technology transfer policies and processes adopted by them. The last section discusses the conclusions of the study and presents recommendations for further studies.

Emergence and (Re) Organization of Public Research Institutions

An analysis of the emergence and consolidation of PRIs in several countries points to a more or less common historical pattern, which took shape from the mid-19th century onward in the context of the second industrial revolution. For Braverman (1980) and Szmrecsányi (2001), this process inaugurated new links between technical and scientific progress, owing not only to the emergence of science-based techniques and the development of technology applied to production but also to the conversion of science and techniques into autonomous and differentiated social activities.

Some studies of the phenomenon of post-war consolidation of public research systems (Jain & Triandis, 1997; Senker, 2000; Potì & Reale, 2000) characterize PRIs as distinct from universities and private research institutions. In this context, PRIs can be understood as organizations that are funded primarily by the public purse, are publicly owned or controlled, and whose principal purpose is to disseminate the research they produce, focusing on goals set externally (by the state) or internally (by the scientific community).

From this perspective, public and private research perform sequential functions: public efforts concentrate mainly on basic and applied research or pursue a specific mission with a significant social return, while private research prioritizes the development, production and commercialization of products and processes. The historical rationale for public S&T institutions is itself grounded in the performance of such functions, which are often represented by unique remits mandated by social and political consensus (Brooks, 1986; Crow & Bozeman, 1998).

During the 1980s an uncoordinated change in their functions and actions led PRIs to look increasingly to planning and management tools that could upgrade their efficiency and effectiveness to levels closer to those of private organizations (Salles-Filho et al., 2000; Salles-Filho & Bonacelli, 2010). This process of reorganization affected PRIs in all sectors of activity and on a global scale.

When the state funds an organization to do research in agriculture or any other sector, it does so not just because otherwise no one else would do it, as prescribed by market failure theory, but because it is required for development, as a policy option, to estimate and influence food supply, or for other reasons deemed relevant. But PRIs can perform this role only if they achieve effectiveness, efficacy and efficiency. This is by no means guaranteed by their legal status or as a direct consequence of the good science they produce. However, it is guaranteed by their capacity to convert good science into benefits for society, which depends on the competencies of each institution in management and relationship building.

According to Salles-Filho et al (2000), three key institutional changes characterized and justified the reorganization processes undergone by PRIs in the 1980s and with particular intensity in the 1990s: (i) changes in the role of the state in capitalist economies; (ii) technical and scientific changes in recent decades; and (iii) growing competition and globalization of markets. Potì & Reale (2000) and Ferreira (2001) also cite a crisis in the public research sector itself, in response to questioning of the trade-off between the resources invested in PRIs and their capacity to achieve the goals desired by government and society.

PRIs reacted differently to these macro institutional changes (Crow & Bozeman, 1998; Salles-Filho et al, 2000; Mello, 2000; Ferreira, 2001; OECD, 2003), but the key element was a redefinition of their roles in innovation systems and their relationships with other actors in such systems, as their scientific and technological activities changed in terms of scope and direction.

An analysis of the process of reorganization of PRIs in agricultural sector beginning in the 1980s points to a number of specific characteristics of this movement, such as a growing tendency to conduct research in response to demand, more decentralization, and greater recognition of the role of agriculture in development policy, with an emphasis on environmental and food security issues (Salles-Filho et al, 1997; Byerlee, 1998; Echeverria, 1998; Roseboom & Ruttan, 1998; Mruthyunjaya & Ranjitha, 1998; Janssen & Braunschweig, 2003). Other significant drivers were changes in demand for agricultural products and changes in national and international agricultural policies in response to trade liberalization. Mendes (2009) discusses the evolution of models for the planning and execution of research and innovation activities in the agricultural sector, noting the passage from a supply-based model (characterized by greater freedom for researchers and the availability of substantial funding, as a reflection of the understanding that research outputs and outcomes would be proportional to investment) to a demand-oriented model, and from there to a mixed model (third generation), which emerged as R&D took on a more strategic role. Salles-Filho & Mendes (2008) interpret this third generation as a phase in which innovation began to find a place on the agenda of research organizations in response to negotiations between actors in the S&T community and the market.

Thus the movement entailed a sophistication of PRIs' management models in agricultural sector via the adoption of more robust tools for strategic planning and for prospecting, monitoring and evaluating research activities, and also via more effective fundraising and human resource management. The main idea is that more recently, innovation has become increasingly important as an element to be considered in the logic of these organizations' activities (Senker, 2000), not only because of a fundraising argument, but also as a way to occupy spaces in which private enterprise is not playing the role originally reserved for them and to participate in an essentially collective game (since networks and systems have become the locus of innovation).

The need to incorporate the logic of innovation should therefore be seen as a new aspect of the reorganization of PRIs in the early years of the 21st century. Albeit partly deriving from the changes that took place in the last two decades of the previous century, this imperative is essentially new because it presents even greater management and cultural challenges than its predecessors.

In sum, although innovation seems a natural route for PRIs to adapt themselves to the new basis for knowledge production and appropriation without changing the rationale for their existence, it has proved no easy task to adjust their organizational and management structures to this new logic. The following sections present case studies of four agricultural PRIs in different countries in order to examine these changes from a practical standpoint.

Research and innovation organization and management models: a case study of four agricultural PRIs

To develop this discussion on the basis of real-world experience, we selected four PRIs in the Americas: the US Department of Agriculture's Agricultural Research Service (ARS), the Research Branch of Agriculture & Agri-Food Canada (AAFC), the Brazilian Agricultural Research Corporation (Embrapa), and Uruguay's National Agricultural Research Institute (INIA).

These PRIs were chosen following a broad survey of the public-sector agricultural research system in the Americas conducted in order to identify organizations whose institutional models and instruments for the management of research and innovation included features that could be considered outstanding. Another criterion was similarity of remits, in the sense that the PRIs selected focus on agricultural research and technology transfer without explicitly referring to rural extension. Table 1 summarizes the main characteristics of these four PRIs.

The four organizations are strikingly different in terms of size and context, yet there are a number of important common characteristics. While the North American institutions date from the 1950s, the South American PRIs are more recent. This does not mean the countries concerned had no previous public agricultural research activities. The dates of their establishment serve merely to identify the time at which the public agricultural research systems in these countries were restructured around national institutes, with the aim of centralizing the coordination of research efforts. Another important similarity is that all four PRIs mainly fund their activities out of their own budgets, although they collaborate with other institutions on many specific projects.

The comparison between the four selected PRIs focused on three main dimensions and was based on secondary data (institutional website and related documents). The first comprised organizational structure, overall organogram and hierarchy, alongside the degree to which activities were centralized or decentralized, and the main programmatic structure used to organize research. The second was the management model, especially the flows and interdependencies characteristic of research planning, programming, execution, monitoring and evaluation. The third comprised technology transfer and its links to research and innovation management.

Organizational Models

All four PRIs are the main research arm of the ministry of agriculture in their respective countries. They all have central units responsible for strategy, planning, management and support, and decentralized units that do the actual research. In the case of INIA and ARS, these decentralized units are named for geographical location and characterized solely by a regional function. Embrapa and AAFC use a different model, with units structured in terms of combinations ranging from regions and ecoregions to cross-cutting themes, products and services. INIA also has technical units located at the same site as the ministry and with cross-cutting functions relative to the decentralized units - three for thematic areas (biotechnology, seeds, agri-weather and other information systems) and two for services (communication and technology transfer, international cooperation). As discussed below, a matrix structure is suggested for all four cases, so that the decentralized units perform their activities within the framework of overarching research programs.

Central units in all four cases relate mainly to communication, information technology, partnerships, and institutional and international relations, besides the usual corporate service functions such as personnel management, budget and financial management, and contract management.

There are also planning and research programming instances, which are responsible for identifying needs and opportunities as well as structuring the portfolio of programs and projects, including resource allocation. It is worth noting that with the exception of ARS these PRIs all separate strategic from tactical instances. The strategic level is mainly concerned with prospecting, identifying demand and opportunities for research and innovation, and drawing up the institution's strategic plans, while the tactical level translates research priorities into the programs and projects that will be executed and monitors scientific and technical activities. Another point in common is the existence of units dedicated to technology transfer. This is discussed in item 3.3 below. It is also important to highlight how the institutions structure their research into programs and projects, as summarized in Table 2. The main differences and similarities evidenced by our analysis are as follows:

• Programs are organized by knowledge areas except in the case of Embrapa, which structures them according to a combination of form and content, such as whether science or technology predominates, the complexity of the research and challenges involved, and the size of the partner network.

• All four PRIs have a typical matrix structure whereby different programs are executed via projects in the various decentralized units.

• All four PRIs have a mixed project portfolio involving internal or external funding, with research being performed internally, evidencing an essentially endogenous bias; only INIA works systematically with the funding of research executed by other institutions (extramural research). • Three of the organizations selected projects by competitive processes (Embrapa, AAFC and INIA for some projects), an effective means of selecting the best projects with a limited budget.

From the basic elements of the four institutions' organizational structures, we now move on to their management models, focusing on processes and flows at the strategic, tactical and operational levels as well as their collegiate governance bodies.

Management Models

An analysis of the four institutions' management models points to significant similarities.They all include systematized formal processes for managing research, including prospecting, planning, prioritization and programming, monitoring and evaluation.

All four PRIs engage periodically in strategic planning (every four to six years) and are guided throughout their trajectory by strategic plans setting goals, targets and actions jointly with the ministries to which they are attached. These strategic plans are drawn up on the basis of consultations with internal and external actors, each involved to a greater or lesser extent in research prioritization. In practice the consultations discuss themes already known to the PRI in one way or another.

Programming processes focus mainly on translating (often freely) the organization's strategic goals, which are always very broad, into programs and projects that act as a framework for research execution. All four PRIs have dedicated teams who define the main challenges and opportunities to be addressed by each program.

Three critical interrelated points stand out with regard to the institutions' planning and programming processes: the first is the capacity to guarantee continuity in identifying requirements and opportunities for the development of projects in the intervals between strategic plans; the second is the effective involvement of external actors in the identification of needs and opportunities; the third is the processes deployed to select projects of interest on the basis of the priorities identified.

Embrapa consults external actors through advisory boards, which include outside representatives and act at the strategic level as well as the more operational level of the units. Projects are selected and prioritized via competitive mechanisms, which begin with requests for proposals based on an institutional strategic agenda. The main criteria are scientific merit and strategic interest. In practice, however, the consultation to outside representatives and the construction of an institutional agenda are not capable to update priorities neither to define programs and projects.

Like Embrapa, AAFC also uses competitive mechanisms to select projects for its portfolio. Each theme is discussed at symposia held from time to time to capture the needs of different actors, assure alignment with the institution's mission and jointly define research lines. Projects for each of the major themes are reviewed by a committee made up of outside researchers, who base their judgment on a set of previously defined criteria (scientific merit, feasibility, and contribution to the creation of public-interest innovations). Where AAFC is ahead of Embrapa is in systematizing the participation of external actors and including innovation potential among the criteria for ex ante project evaluation. Even though it is not clear how this practice actually changes the portfolio of programs and projects.

INIA has regional advisory boards and thematic working groups consisting of outsiders. These bodies provide permanent support, consultation and advice to national research programs and the managers of regional units. They are designed to act as radars or antennas that capture and anticipate regional needs in terms of research, diffusion, transfer and development. These outside collegiate bodies engage in systematic prospecting to orient the composition of the project portfolio, as well as monitoring and evaluating projects. Projects are selected on the basis of the priorities identified, with prior evaluation by outside peer review based on criteria relating to potential economic, social and environmental impact as well as scientific quality. More recently, like Embrapa and AAFC, INIA has introduced competitive mechanisms for project selection under the aegis of research development funds. In addition to the collegiate bodies that are part of the organizational structure, INIA also participates actively in Industry Councils (Mesas Sectoriales) and Regional Innovation Consortia (CRIs) as another way to assure continuity in the identification of needs and opportunities and their conversion into research projects that effectively meet those needs and opportunities.

ARS operates similarly to AAFC, periodically holding workshops attended by its own researchers and external actors to discuss specific action plans for the various national programs that serve as a framework for research project selection. ARS has a dedicated unit to evaluate the scientific quality of projects (which organizes scientific panels with outside members), as well as measuring their relevance in terms of alignment to the institution's mission and strategic plan, and the needs of US agriculture. These mechanisms concern both project selection and project monitoring over time.

Having defined the portfolio of projects to be executed, the institutions establish research monitoring and evaluation processes. As noted above, in the case of ARS and INIA these are very close to the project selection process since the same unit supports both project selection and project monitoring and evaluation.

All four PRIs monitor and evaluate projects. In the case of monitoring some are more systematic than others, but generally speaking it consists of verifying the extent to which the projects comply with the goals and targets set, either using a logical matrix approach (what was promised versus what was achieved) or via peer review. Except in the case of Embrapa, evaluation takes the form of ex ante assessment of potential impact rather than ex post impact evaluation.

The key point to be derived from this is that these institutions' evaluation efforts have been essentially intramural, i.e. actual versus expected results or targets achieved versus targets planned and programs. Systematic efforts have not been made to measure outcomes, i.e. the impact on the outside world. Even Embrapa, which has a long tradition in impact evaluation, uses methodologies based on successful technologies, so that they do not permit a comprehensive evaluation of return on investment. As a result there are no indicators that effectively represent the impact of research projects, although there are significant quality assurance efforts based on the assessment of merit (ex ante) in accordance with the canons of peer review within the scientific community, but not those of technological or innovation evaluation.

Technology Transfer

In line with the institutional changes that have occurred, all four PRIs have created structures and mechanisms to foster technology transfer and innovation with the aim of increasing the impact of their research and the benefits to society.

They have done this by including offices or units in their organizational structures with the remit of implementing activities for the transfer and commercialization of technology resulting from the research performed within the institution concerned.

While varying in certain specific ways from one institution to another, these structures have the following basic functions:

• Developing and coordinating technology transfer policy;

• Negotiating and drafting collaboration research agreements with other actors in the system, public or private, local or foreign;

• Identifying potential collaborators and organizing databases on partner institutions, projects, programs and agreements or contracts;

• Negotiating technology transfer agreements;

• Identifying knowledge that can be protected and the demands coming from research institutions, universities, rural extension agencies, industry, the third sector and government;

• Providing researchers with strategic guidance on intellectual property protection and management of IP rights intramurally or via outside contractors;

• Executing licensing agreements, including in some cases business risk assessment (case of AAAFC) and license performance monitoring (cases of ARS and AAAFC);

• Developing, implementing and coordinating the institution's marketing strategies to facilitate technology transfer;

• Performing, coordinating and supporting research into methodologies and criteria for surveying, categorizing and prioritizing technology requirements - particularly strong in the case of Embrapa and INIA.

In some cases these functions are not centralized but performed by more than one unit. In the case of Embrapa, they are distributed between the recently created Technology Transfer Department (DTT) and the Technological Innovation Office (AIT), both of which report to the CEO. Embrapa Technology Business Unit (SNT) is a decentralized unit that also participates in this process. DTT works with SNT on planning, coordination and evaluation of technology transfer processes and actions, collaborating where necessary with R&D, communication, and agricultural technology transfer organizations, as well as partnering with other institutions in the National Agricultural Research System. AIT is responsible for coordinating and overseeing negotiations with groups interested in innovative products, managing the institution's IP rights, and prospecting for technological and market opportunities, which involves evaluating technologies already created and finding ways to commercialize them in order to build a technology portfolio. It seems that Embrapa has created different internal departments and areas to deal with the innovation processes. The Institution separates TT from innovation and from technological business showing that the theme of innovation is still in process of construction.

AAFC also has a unit dedicated exclusively to innovation, which promotes the formation of clusters to facilitate the use and appropriation of the institution's knowledge and technology by society.

In Uruguay, technology diffusion and identifying requirements are functions of INIA's Communication & Technology Transfer Unit. The other functions are the responsibility of a unit called Technology Linkage Unit (Gerencia de Vinculação Tecnológica), set up in the 2000s to capture, coordinate, support and promote academic and business partnerships for the development of high-impact agricultural research, commercialize the institution's products, processes and services, and manage its IP rights.An example of this is the recent creation of Regional Innovation Consortia (CRIs) to coordinate research, technology transfer, rural extension, training communication, diagnosis, and assistance for local development plans, among other functions. Formal agreements stipulate the terms and conditions governing the human, physical and financial resources contributed by each of the members of the public-private partnerships (PPPs) established.

This analysis highlights one of the critical elements in the organizational models adopted by the PRIs studied, which is precisely the separation of research management from the management of technology transfer and innovation. This approach reproduces the linear model of innovation, in which creation, adoption and diffusion are distinct and sequential stages.

It is worth stressing in this regard that the long-term competitiveness of these four PRIs depends strongly on the transition from being R&D organizations to being RDI organizations. This means moving beyond the linear model to adopt an approach based on the interactions between knowledge creation and knowledge use.

Besides units dedicated to technology transfer and innovation, all four PRIs also have institutional policies governing technology transfer. Thus AAFC has clear rules on the registration of technologies and licenses, requiring for all forms of licensing a business plan that describes how the technology will be developed and commercialized as a means of assessing the performance and impact of the license.

Furthermore, both AAFC and ARS publicize the technologies available for licensing on their institutional websites, with detailed information to assist anyone interested in applying, including a full description of the technology and potential applications including market, limitations, third-party rights etc. AAFC issues requests for proposals in some cases. ARS provides online forms for applications to license patents and pending patents, plant varieties and biological materials. It does not manage licensing as a competitive process and negotiates fees, royalties and license types on a case-by-case basis.

INIA also has a policy to assure legal protection of knowledge, especially with regard to IP rights to plant varieties and their multiplication and commercialization, as well as collecting royalties for their use (Delpiazzo, 1994). The royalties collected are used for investment in research in the area of the product that earned them. Like AAAFC, INIA publicizes its "technological products", with full technical descriptions and indications of end-users. In 2010 it published a catalogue of research products from several sources, like those issued by INTA in Argentina and Embrapa in Brazil. The catalogue emerged from broad internal consultation of program heads and lead researchers. The catalogue production process also served as an opportunity for preliminary evaluation of the outputs of each project included in the Medium-Term Implementation Plan for 2007-11.

Conclusions

The analysis showed that all four PRIs have evolved in terms of consolidating and enhancing their management models to prioritize innovation. The highlights of this evolution are (a) consolidation of the institutional strategic planning process; (b) a research management structure based on thematic programs, programs organized by type of research or collaborative arrangements; (c) a matrix structure in which decentralized units in charge of execution interact in a cross-cutting manner with the various institutional programs; (d) a mixed portfolio of projects with internal and external funding; (e) growing use of competitive mechanisms for project selection; (f) participation to a greater or lesser extent by different categories of actors from the national agricultural innovation systems in prospecting, prioritization and definition of research themes; (g) adoption of intellectual property and technology transfer practices and policies with the aim of commercializing and diffusing the knowledge and technologies produced.

In sum, the institutions analyzed are effectively reorganizing to promote innovation processes by creating tools, mechanisms and instances that build stronger links between R&D and the needs of all sectors of society, rather than agriculture only. The main results point to sophistication of their planning and programming processes, and to more integration between the viewpoints of the key actors in agricultural innovation systems and the requirements of society. This is especially well designed and implemented in the Uruguayan case.

Nevertheless, much remains to be done. Innovation has not yet been fully integrated into research efforts and is still seen as a phase on which to focus after research is completed. In addition, it is not being adequately measured.

Specific structures and policies have been created for technology transfer but they are separate, and in some cases isolated, from the R&D and partnership structures and policies, without a cross-cutting axis that takes care of innovation processes and includes the perspective of S&T knowledge creation and demand.

There are efforts to measure research impacts and outcomes but they are timid with regard to the use of methodologies and indicators to evaluate society's effective appropriation of the knowledge and technology created.

Another point refers to the fact that all four PRIs present excessively endogenous research execution structure. They undertake few extramural research funding initiatives, although they do engage strongly in partnerships and research arrangements with other institutions. Decreasing reliance on endogenous funding means more sharing of competencies and enhanced engineering of partnerships in project execution (more and better exploration of make-or-buy opportunities in all variations - what to do alone, what to do collaboratively, what to "buy in").

Finally, one important conclusion refers to the fact that these institutions are more and more dedicated to produce club goods once they are engaged in protecting IP rights and trying to participate in a world where public goods are not an intention, but a result of the ways a technology or a knowledge will or will not be appropriate by the innovation system.

In short, the key point for agricultural PRIs is no longer contributing to the production of knowledge and technology but incorporating efforts and competencies relating to the use of the knowledge and technology they produce. They not only have to produce "relevant knowledge" but they also have to compete with other actors, driving themselves to a club-like behavior, normally quite similar to the rules of private research organizations.

This entails not simply merging the creation and diffusion phases but integrating them with the innovation processes and with the innovation system, which requires management capabilities that change the ways in which research is designed, performed and evaluated.

References

BRAVERMAN, H. (1974). Labor and Monopoly Capital: The Degradation of Work in the Twentieth Century. Monthly Review Press, New York.

BROOKS, H. (1986). National Science Policy and Technological Innovation. In: Landau, R., Rosenberg, N. (Eds.), The Positive Sum Strategy: Harnessing Technology for Economic Growth. National Academy Press, Washington. pp. 119-167.

BYERLEE, D. (1998). The search for a new paradigm for the development of national agricultural research systems. World Development, 26(6), 1049-1055.

CROW, M., Bozeman, B. (1998). Limited by design: R&D laboratories in the U.S. national innovation system. Columbia University Press, New York.

DELPIAZZO, C.E. (1994). Tratamiento Jurídico de los Recursos Fitogenéticos: política institucional del INIA en materia de innovaciones biotecnológicas. Serie Temas Institucionales, n. 4. INIA, Montevideo.

ECHEVERRÍA, R. (1998). Will competitive funding improve the performance of agricultural research? Discussion paper, 98-16. ISNAR, The Hague.

FERREIRA, C.R. (2001). Tendências de reorganização da pesquisa: um estudo a partir de experiencias internacionais. Master's dissertation. Department of Science and Technology Policy, University of Campinas, Campinas.

JAIN, R.K.; Triandis, H.C. (1997). Management of research and development organizations: managing the unmanageable. John Wiley & Sons, New York.

JANSSEN, W.; Braunschweig, T. (2003). Trends in the organization and financing of agricultural research in developed countries: implications for developing countries, Research Report, 22. ISNAR, The Hague.

MELLO, D. L. (2000). Análise de processos de reorganização de institutos públicos de pesquisa do estado de São Paulo. PhD thesis. Department of Science and Technology Policy, University of Campinas, Campinas.

MENDES, P. J. V. (2009). Organização da P&D agrícola no Brasil: evolução, experiencias e perspectivas de um sistema de inovação para a agricultura. PhD thesis. Department of Science and Technology Policy, University of Campinas, Campinas.

MRUTHYUNJAYA, Ranjitha, P. (1998). The Indian agricultural research system: Structure, current policy issues and future orientation. World Development, 26(6), 1089-1101.

OECD. (2003). Governance of Public Research: toward better practices. OECD Publications Service, Paris.

POTI, B., Reale, E. (2000). Convergence and differentiation in institutional change among European public research systems: the decreasing role of public research institutes. Science and Public Policy, 27(6), 421-431.

ROSEBOOM, J.; Ruttan, H. (1998). The Transformation of the Dutch Agricultural Research System: An Unfinished Agenda. World Development, 26(6), 1113-1126.

SALLES FILHO, S., Bin, A., Gianonni, C., Mendes, P.J.V., Rio, C. (2009). GCARD Regional Review for Latin America and Caribbean. GFAR, Roma.

SALLES-FILHO, S., Albuquerque, R., Mello, D. (1997). Novos rumos da pesquisa agrícola e agroindustrial. In: Almeida, J., Navarro, Z. (Org). Reconstruindo a agricultura: idéias e ideais na perspectiva do desenvolvimento rural sustentável. Editora da UFRGS, Porto Alegre. pp. 189-203.

SALLES-FILHO, S., Bonacelli, M. B. M. (2010). Trends in the organization of public research institutions: lessons from Brasilian case. Science and Public Policy, 37(7), 193-204.

SALLES-FILHO, S., Mendes, P.J.V. (2008). Sistemas Nacionais de Pesquisa Agrícola: àmbito Federal. In: Silva, A.G., Albuquerque, AA.C.S. (Eds.) Agricultura Tropical: Quatro Décadas de Inovagóes Tecnológicas, Institucionais e Políticas. Embrapa Informagóes Tecnológicas, Brasília.

SALLES-FILHO, S.; Albuquerque, R.; Szmrecsányi, T.; Bonacelli, M.B.; Paulino, S.; Bruno, M.; Mello, D.; Corazza, R.; Carvalho, S.; Corder, S.; Ferreira, C. (2000). Ciencia, tecnologia e inovação: a reorganização da pesquisa pública no Brasil. Editora Komedi, Campinas.

SENKER, J. (2000). Introduction to a special issue on changing organisation and structure of European public-sector research systems. Science and Public Policy, 27(6), 394-396.

SZMRECSÀNYI, T. (2001). Esbogos de história económica da ciencia e da tecnologia. In: Soares, L.C. Da Revolução Científica à Big (Business) Science. Hucitec/Eduff, São Paulo.

Selected February 11, 2013