Assessing economic impact of research and innovation originating from public research institutions and universities—case of Singapore PRIs

This section presents a descriptive analysis of our consideration set of 275 articles and reviews the dimensions of public research impact assessment posited in these studies. Figure 1 shows that 275 articles were published in a total of 50 journals.

Given that impact studies may be undertaken at various levels of analysis, ranging from individual through institutional to national, the range of journals publishing such studies is correspondingly wide, from the fields of business economics to public administration. However, a closer examination of the journal titles shows that the distribution of journal titles is rather skewed, with three journals accounting for more than half of the journal population: Research Policy (35 %), Journal of Technology Transfer (12 %) and Technovation (5 %). While Research Policy has consistently topped the chart, Journal of Technology Transfer saw an explosion of publications in this area in the recent decade. There may be several explanations for this phenomenon. Using the lens of life cycle model to the development of new inquiry topic, the topic of technology transfer was in the nascent stage of development. It is noteworthy that following the US Bayh–Dole Act of 1980 and similar developments in other innovating economies, which encouraged universities and PRIs to patent their inventions with government funds, a rise in technology transfer from public research to the industry spear-headed a concomitant growth in the literature of technology transfer, particularly in the USA and Europe. As new topics of inquiry gain attention, the editorial teams of more prestigious journals that might be initially sceptical of their validity became more willing to publish them or new journals such as the Journal of Technology Transfer being developed to carry the scholastic conversations (Kuhn 1962). During the period of 1973–2013, we observe an increasing volume of impact studies, as presented in Fig. 2.

Our review of public research and innovation impact studies shows that interest in this topic started back in the 1970s, when the US tertiary institutions were recommended to assess the direct and indirect consequences of their investment and expenditures on their regional economy by the American Council on Education (Goldstein 1989).

In the 1980s, the national innovation system (NIS) concept gained attention with the works of Freeman (1987) in Japan and Nelson (1993) in the USA in the area of public-funded research. Their works highlighted the significant roles that universities, PRIs and government agencies play in pursuing technological progress and contributing to economic growth. The public policy makers’ rising expectations of academic innovation impact on the industry and society may be seen in the proliferation of studies on this topic as politicians, economists and technology management scholars questioned about the diverse aspects of public research impact ranging from the rationale and feasibility of measuring such impact to the conceptual framework and methodology of monitoring and managing it.

Brownrigg’s (1973) case study of UK University of Stirling found that its impact on the local economy in terms of income and employment might be marginal due to the significance of other factors such as local industrial structure and its ability to promote trade and investment flow. However, Brownrigg highlighted the limitations of traditional input-out modelling used by the economists in such impact studies of local economy. A similar case study approach coupled with university panel data was adopted by Keane and Allison (1999) in their study of Australian University of Sunshine Coast, Queensland, to establish the strength of university-industry connection and the degree of its “embeddedness” in the region.

In the USA, Bozeman and his colleagues developed the research value mapping (RVM) approach to assess the socio-technical impact of applied R&D projects with the Department of Energy (Bozeman and Kingsley 1997; Bozeman and Klein 1999). Using a combination of qualitative (case studies) and quantitative techniques (data developed from case studies), Bozeman and his team spent a considerable amount of effort to track the knowledge flow of R&D projects and model the possible project outcomes such as technologies patented or licensed, new companies created, new or improved products or processes developed or enhanced human capital. The RVM was found to work well for summative evaluation; however, its adoption beyond the USA has been limited due to the extensive resources required to trace and perform value mapping. According to Hicks et al. (2002) report submission to the Japanese National Institute of Science and Technology Policy, the RVM’s identification of projects with significant milestone achievement to be units of analysis could provide administrative convenience from the funding perspective but did not reflect how the researchers view their work and capture other non-economic payoffs such as the development of human capital (Georghiou and Roessner 2000).

Jaffe (1989, p. 957), on the other hand, used US state-level time-series data to present the significance of “geographically mediated spill-overs from university research to commercial innovation” in the form of corporate patents, particularly in the drugs, chemicals and electronics industrial sectors. These findings were corroborated by Fischer and Varga’s (2003) work on 72 Austrian districts using knowledge production function framework, as well as Beugelsdijk and Cornet’s (2002) study of 1510 Dutch firms, who found that their proximity to technical universities had significant spill-over effects on the firms in the form of new products. Mansfield’s (1991) study of 76 major US firms attributed 11 % of their new products and 9 % of new processes to academic research.

Harhoff’s (1999) econometric analysis of German engineering industry sectors yielded similar evidence of regional spill-over effects from academic institutions on the sectors in the form of new business formation. However, Lööf and Broström’s (2008) study in Sweden found significant academic impact only on large manufacturing firms in the form of innovative sales, while the average service firms did not seem to benefit from university collaboration.

More recent studies are also found in Spain by Barajas et al. (2012) who used the CDM (Crépon et al. 1998) model to measure academic spill-over effects in the form of sales per employee; and Italy by Colombo et al. (2010) who applied the resource-based theory (Wernerfelt 1984) to assess innovation impact in terms of spin-offs.

In France, Bach and Llerena (2007), in their study with BETA, urged to look beyond traditional technology innovation indicators, such as R&D expenditures and licensing revenue, to capture the interaction dynamics among the institutes. It is noteworthy that BETA of France and NIST of the USA had introduced their respective methods in the 1990s to assess the impacts of large-scale R&D projects such as the European Space Programme (Bach and Lambert 1992) and Measurement and Standards Laboratory Programme (Tassey 1999). These methods were subsequently adopted by other countries such as Japan in their study of techno-economic impact of large-scale R&D projects (Kondo 2012). While these methods have been useful for estimating the impact of R&D projects, their limitation on the timing and size of the impact studies has been highlighted as the projects selected for studies must have either completed or achieved some significant benefits. As the time elapsed between project start-up and project benefit realisation might range two to seven  years, based on past studies (Tassey 1999), BETA and NIST used these methods only for large-scale projects, while employing other forms of intermediate indicators, such as new companies started and product or process innovation for periodic and more pragmatic assessment of their innovation impact.

To build a comprehensive multi-dimensional framework, we applied the R&D-based endogenous growth theory (Romer 1990) and Triple Helix (Etzkowitz et al. 2005) to model the flow of knowledge among the major stakeholders of the industrialised economies: universities and PRIs, firms and industry, as well as the community and society. The endogenous growth theory (Romer 1990) postulates that knowledge and technology, unlike physical objects, are characterised by increasing returns, thereby driving economic growth. The theory forms the basis of knowledge-based economy and emphasises the critical role that the economic processes of knowledge creation and diffusion play in shaping growth of firms, industries, communities and nations. The Triple Helix, on the other hand, underscores the central role of universities and its relations with the industry and government in advancing the knowledge-based society (Etzkowitz et al. 2005).

To synthesise the data gathered into the framework, we categorised the dimensions of impact assessment which emerged from our literature review by the different types of knowledge generated and associated output indicators from the various stakeholders: (1) inventive output of universities/PRIs, (2) commercialisation output of universities/PRIs by their TTOs, (3) innovation of industry/firms and (4) community/economy. Figure 3 shows the distribution of impact study papers by categories 1 to 4.

While the majority of papers (84 %) fall into only one of the four categories, some articles belong to more than one category. For example, some studies (e.g. Derrick and Bryant 2013) used both the inventive output of the universities (publications—category 1) and commercialisation output of universities/PRIs (licenses generated—category 2) to measure the impact of research incentives on Australian medical research organisations, thereby falling into in category (1/2) as illustrated in Fig. 3.

Figure 4 presents the conceptual framework depicting the processes and associated output captured by studies among these four stakeholder output categories.

The categories are represented as blocks and the knowledge flows among the blocks depicted by bilateral arrows. These bilateral flows among the blocks are better representation of knowledge flow directions than the traditional linear knowledge flows from universities/PRIs through the firms/industry to the economy as illustrated by most studies in the current literature (Gulbrandsen and Rasmussen 2012). In practice, there is much knowledge exchange between universities/PRIs and the industry, as the inventive output from the former could be inspired by the problems encountered by the latter through research collaboration. In a similar vein, the commercialisation output of the former could be shaped and refined by the requirements of the latter through the open innovation approach adopted by firms (Chesbrough 2007). In addition, universities/PRIs could have direct impact on the economy through their direct expenditures and employment. This direct interaction between universities/PRIs and economic development is represented in our conceptual framework by enveloping universities/PRIs/TTOs (block 1 and block 2) and industry/firms (block 3) with the larger community/economy (block 4). The knowledge flows are therefore indicative of the four phases of the RIE value chain.

It is important to note that this framework is distinct from the chain-linked model postulated by Kline and Rosenberg (1986), which focuses on the relationships among research, invention, innovation and production within the boundaries of an organisation. The Impact Assessment Framework, in contrast, looks beyond the organisational boundaries to access complementary assets and open innovation. The categorisation of block 1 and block 2 is supported by Gulbrandsen and Rasmussen (2012), who made similar distinction between inventive output of universities/PRIs and their commercialisation output, articulating that the former measures the potential rather than the actual commercial value of public science. Block 3 is consistent with the findings of Crépon et al. (1998), who distinguished between innovation input (R&D) and innovation outcome (product and process innovation). Block 4 is corroborated by the work of Goldstein and Renault (2004, p. 735), who advocated that universities could have potential impacts including “direct and indirect spending impacts”.

In the case of Singapore PRIs, all the indicators highlighted in blocks 1 and 2 were used to measure their inventive and commercialisation output for tracking progress and benchmarking with local and international counterparts, as indicated in Fig. 5.

To assess the commercialisation impact of PRIs’ research at firm, industry and economy levels, the PRIs actively tracked since 2006 the RICV of the intellectual property (IPs) generated from the licensees’ commercialisation of the IPs and aggregated them at the various levels. RICV was captured for two purposes: (a) obtain the actual commercial value generated by the IP and (b) validate their pre-deal assumptions of projected cash flows as feedback mechanism.

According to the descriptive statistics in Table 1, more than 50 % of the licensing agreement transactions were with small and medium enterprises (SMEs), 40 % with multi-national corporations (MNCs) and large enterprises and the remaining with start-ups. About one third of the licensing agreement transactions came from the biomedical sciences sector, one third from info-communication sector and the rest from engineering, manufacturing, electronics, materials, chemicals and other sectors.

Our regression results for H1 are presented in models 1 and 2 of Table 2. Model 1 is a baseline model that examines the variances contributed by the control variables. Model 2 describes how IP licensing fees affects a firm’s propensity to repeat licensing pertaining to H1. Model 2 does not appear to be an improvement over model 1, as evident from the similar pseudo R square values, Cox and Snell R square (0.08) and Nagelkerke R square (0.13), as well as the percentage of correct classification (78 %). However, with a coefficient (b = −0.23, p < 0.1), IP licensing fees is found to be a significant but negative predictor of the firm’s propensity to repeat licensing, indicating support for H1 that firms that incur higher IP licensing costs have lower propensity to repeat their IP licensing transactions, thereby reducing academia innovation impact on firms.

The regression results for H2 are presented in model 3 of Table 2. Model 3 is an improvement over the baseline model 1, as seen from the increase in the Cox and Snell R square (from 0.08 to 0.17), Nagelkerke R square (0.13 to 0.27) and the percentage of correct classification (78 to 82 %).

With a positive coefficient (b = 1.67, p < 0.05), RICV is established to be a significant and positive predictor of the firm’s propensity to repeat licensing, thereby validating H2 that firms that attain higher levels of RICV with universities/PRIs have higher propensity to repeat their license transaction.

To explore possible interaction effects between IP licensing fees and RICV, model 4 is a full model demonstrating the main and interaction effects of all variables, including the interaction term between IP licensing fees and RICV. The interaction term is found to be not significant. In models 1 to 4, some control variables such as firm size and engineering and manufacturing sector seem to be significant. After controlling for them, there is no change in the significant contributions of the independent variables to the dependent variable, confirming support for H1 and H2. Combining the results of H1 and H2, we can establish that RICV is a better indicator than IP licensing fees in predicting a firm’s propensity to repeat licensing and indicating academic innovation impact on firms. With more than 100 cases per predictor variable, these models satisfy Field’s (2005) guidelines of minimum 10 cases per predictor for reliable regression results. With a variance inflation factor (VIF) value of 1.234 that is lower than the cut-off value of 10 recommended by Chatterjee and Price (1991), the probability of multi-collinearity issues affecting the findings is relatively low.

Combining the results of H1 and H2, we establish that while IP licensing fees and RICV have opposite effects on firm’s propensity to repeat licensing with PRIs and academic innovation impact on them. The findings are consistent with Atuahene-Gima’s (1993) findings that firms’ ITL propensity was determined by their perceived value, namely, their benefits relative to their costs.

Although this study involves PRIs, its results may be seamlessly extended to universities for several reasons. First, since the implementation of the US Bayh–Dole Act and its equivalents in other knowledge-based economies, PRIs and universities in most countries have been charged with similar missions to create socio-economic impact through promoting knowledge generation and transfer to the industry (Link et al. 2011). Second, although PRIs are expected to work on more applied research while universities on basic research, both share similar strategic roles in setting future R&D directions, especially in areas critical to their nations, e.g., energy or climate. As a result of their longer planning horizon, their technologies are at an earlier stage of development compared to the industrial R&D centres, making their time to commercialisation longer and more risky than that of their industrial counterparts (Link and Scott 2013). As both universities and PRIs share similar goals, roles, pressures and challenges in their technology transfer processes, many studies in the technology transfer literature have treated them as a homogeneous group and extend their findings across them (Park et al. 2010).

In the first block of the framework in Fig. 4, the studies generally adopted the approach of using the inventive output of universities/PRIs as proxy indicators to estimate their social and economic impact. The notion of knowledge as key driver for the competitive advantage of firms and nations has its roots in the literature of economics building on Romer’s endogenous growth theory (Romer 1990; Nonaka and Takeuchi 1995; Grant 1996). The capacity and capability to develop novel ideas and translate them into new products and processes for economic growth form the knowledge base of an economy (Huggins and Johnston 2009). These innovation development capabilities are increasingly being associated with universities and PRIs. As knowledge becomes publicly accessible either in the form of spill-overs or diffusion, it is widely regarded as a desirable outcome for the greater good of the society, spurring greater innovation and economic growth (Fischer and Varga 2003). Table 3 provides a summary of key studies using inventive output indicators.

As centres of knowledge production, universities/PRIs consistently engage in inventive process, where researchers develop ideas that could be novel and inventive. These inventions are then submitted as required by the Bayh–Dole Act or equivalent in the form of invention disclosures for evaluation by the TTOs for the appropriate intellectual property (IP) strategy to pursue. For invention disclosures that satisfy both internal and external patenting guidelines, the TTOs would proceed to file patent applications, while the researchers would submit their findings for publications. Some high-technology industries may find these publications or patents relevant and cite them or even participate in research collaboration with the universities/PRIs in the course of their R&D work.

Given the significant government R&D expenditures in universities and PRIs to develop the knowledge base of the economy, the studies of this category frequently built on the total factor productivity or knowledge production function frameworks to examine the various outputs in the inventive process as indicators to determine the economic benefits directly or indirectly generated (Jaffe 1989). Building on Cobb–Douglas production function, Pakes and Griliches (1984) pioneered the development and use of the full model of knowledge production function and the final output production function. In their study, they used R&D expenditures as knowledge creation investment and patent counts as economically valuable knowledge produced. Besides economic impact, many researchers have also adopted bibliometric approaches to assess the degree of international scientific influence using citation, mapping and network analysis of papers and patents (Tornquist and Hoenack 1996; Spencer 2001). Others have studied co-authored publications in their bibliometric examination of public-private research collaboration (Abramo et al. 2009a). While citation indicators may reflect productivity and impact, co-authorship indicators may illustrate informal network access, knowledge diffusion and skill transfer.

Cohen et al. (2002) found that publications are important knowledge channels from the public sector to the manufacturing sector. This is supported by Agrawal and Henderson’s (2002) interview of MIT faculty, who established that publications were two-and-one-half times more significant than patents as knowledge channels. Jaffe (1989) found that the proximity between universities/PRIs and firms could have mediating effects on spill-overs from public-funded research to corporate patents. Subsequent studies suggested similar association between patent citation patterns (as indicator of “economic benefit”) and geographical proximity in Austria (Fischer and Varga 2003) and in the USA (Jaffe et al. 1993).

Building on the RVM project, Dietz and Bozeman (2005) discovered significant differences between publication and patent productivity in their socio-technical human capital project. Career spent in industrial settings was found to be negatively related to publication productivity but positively related to patent productivity. In a similar vein, the source of industry support was identified to be unrelated to publication productivity but positively related to patent productivity. These findings suggest that governments that aim to improve the commercial relevance of their public-funded research should encourage “revolving door” policy between academia and industry in their development of science and technical human capital to enhance their university/PRI inventive output.

Pakes and Griliches (1984) established a strong association between R&D and patent counts at the firm and industry levels to demonstrate that patents are appropriate measures of inventive output. Although patent and patent citations have frequently been used as economic indicators, there is concern that patents do not provide the complete picture of economic contribution by universities/PRIs (Fischer and Varga 2003). First, patents do not capture all forms of knowledge, such as codified knowledge (trade secrets and copyrights) and tacit knowledge (know-how and experience), which is important for spill-over. Second, patent count is dependent on the intensity of patenting activity, which is in turn determined by the patenting policy of the institutions. Therefore, patent counts and citations are not necessarily reflective of the commercial value of underlying IP.

According to a study by Intellectual Ventures in 2008, although 60 % of the patents were developed by academia, PRIs and individual inventors, they contributed to less than 1 % of the licensing revenue. On the other hand, large enterprises which were the originators of 40 % of the patents received more than 99 % of the licensing revenue (Hagiu et al. 2011). The stark contrast in licensing revenue between these groups that is disproportionate to that in patent origination signals critical issues with patents as economic indicators.

This demonstrates the urgency for more commercialisation-related indicators that policy makers can implement if they are committed to encouraging more firms to license and commercialise public science (Cheah and Zalan 2013).

The second block of the Impact Assessment Framework in Fig. 4 dwells on the output of commercialisation process that TTOs use in taking IPs from their universities/PRIs to the market. Since the 1990s, there has been an increasing awareness that technological innovation may not be confined within the boundaries of an organisation. It has been widely acknowledged that technology is an essential resource that can be converted into a capability and competitive advantage for firms and nations. Firms have been giving more attention to the sourcing of external knowledge to complement and enhance their existing R&D stock and the acquisition of external technology as part of their R&D process. A summary of key studies using university/PRI commercialisation output indicators is listed in Table 4.

According to Hemmert (2004), external technology acquisition takes place by (a) licensing technology, (b) research collaboration with other organisations and (c) purchasing technology from other organisations. TTOs typically do not sell the technologies originating from universities/PRIs, but prefer to assign non-exclusive licenses to firms if the inventions are broad in scope and wide in applications across industries, so as to encourage product development in more fields for maximum economic impact. Notwithstanding this, TTOs may consider exclusive licensing terms for inventions that require substantial investments for commercialisation. Nelson’s (2009, p. 994) comparison of IP licenses, publications and patents found IP licenses to be the most reliable indicator of “downstream knowledge utilisation” as the “competing economic interests” between licensors and licensees ensure that the licensing agreement is a fair representation of industry demand. Besides licensing IPs to existing firms, TTOs may facilitate the formation of spin-offs by providing technology and business incubation services to their researchers.

As both developed and developing nations put in place the appropriate science, technology and education policies to facilitate technology commercialisation and academic entrepreneurship from universities/PRIs, there has been much scrutiny on their commercialisation output, namely, the number of IP licenses signed, the IP licensing revenues collected and the number spin-offs generated.

To date, more than 14 periodic surveys had been launched to capture such output indicators across different regions such as the USA (AUTM 1991–2012), Canada (AUTM 2000–2012), Europe (ProTon 1991–2012) and Singapore (A*STAR 2002–2011). The data collected from these surveys form the dataset for many studies in this literature strand. For example, AUTM data were used by Di Gregorio and Shane (2003) to show that the rate of university spin-offs depends on inventors’ quality and equity stake in the spin-offs. Calls have been made to both AUTM and ASTP to standardise their variable definitions and methodological approaches to facilitate comparability of PRI indicators across the studies. However, there is a dearth of impact studies in the Asia.

One frequently used theoretical framework in these studies is the resource-based theory (Wernerfelt 1984), which postulates that a firm’s performance is determined by its resources. Barney (1991, p. 117) defined resources as assets that are specific to the firm, “valuable, rare, imperfectly imitable, and non-substitutable”. Building on the resource-based theory, O’Shea et al. (2005), Colombo et al. (2010) and Powers and McDougall (2005b) established that the quality of researcher and research has significant effect on spin-off formation and performance, which were commercialisation output indicators used to estimate public research economic impact.

While licensing number and revenue are useful and relevant indicators to measure commercialisation process efficiency, several issues have been raised about their adequacy in reflecting the economic benefits of public-funded research. First, as license data collected by TTOs only reflect commercialisation activities during the licensing period, which is capped between patent grant date and expiration date, they do not capture any further innovative activity that may continue to impact the economy beyond the licensing period. Second, it is widely recognised that the pathway of commercialisation from invention (that the firm has licensed from universities/PRIs) to innovation (that the firm can generate sales in the form of products and services) requires significant investment as it involves early-stage technology development. This pathway is often alluded to as the valley of death due to the high failure rate, and the role of the public sector in bridging this valley has been much debated upon in the UK and USA (Link and Scott 2013).

The current issues with commercialisation output indicators call for other indicators that would better reflect the downstream innovation process taking place in the firms to provide a more complete picture of public science impact on the industry.

The third block of the Impact Assessment Framework focuses on the innovation of industry or firms. Table 5 summarises the studies using firm’s innovation input and outcome indicators.

Some of the studies referred to the knowledge production function (Jaffe 1989). Acs et al. (1994) built on knowledge production function to ascertain that small firms could benefit (with increase in innovation count) from the knowledge of universities and the R&D centres of large firms. Varga (2000), on the other hand, applied the Griliches-Jaffe knowledge production function framework to establish that firms located in close proximity to the research university can benefit economically with an increase in their number of innovative products.

In 1998, Crépon et al. (1998) integrated both Cobb–Douglas production function and knowledge production function to develop the CDM model, which depicts two key relationships pursuant to a firm’s decision to invest in innovation. The first relationship describes the link between innovation input (primarily R&D) and innovation output (product or process innovation) as accepted in the knowledge production function (Pakes and Griliches 1984). The second relationship established the connection between innovation output and productivity based on Cobb–Douglas production function. Since then, many studies have built on the CDM model using labour productivity (sales per employee) in Spain (Barajas et al. 2012) and in the Netherlands (Lööf and Heshmati 2002) and turnover growth in Italy (Evangelista and Vezzani 2010).

Based on the CDM model, we have classified the following indicators as input to the innovation process of firms: (a) corporate R&D stocks, (b) innovation expenditures, (c) corporate patents, (d) investment in building and equipment for R&D and (e) induced investments. In particular, Pressman et al. (1995, p. 30) surveyed a sample of MIT licensees by collecting information on their induced investment defined as “Money spent developing new products and efficient ways to produce and market these products. It excludes the costs of producing (or investment required to produce) mature products”. From the survey, they extrapolated their data to estimate an economic impact of US$922 million.

Using a similar approach, Kramer et al. (1997) estimated that University of Pennsylvania’s licensees created an economic impact of US$4.6 billion in the form of induced investments. Although Pressman et al. (1995) and Kramer et al. (1997) have used induced investments to measure economic impact, these values are actually input to the firm’s innovation process that should be distinguished from its innovation outcome. According to Eurostat (2012), innovation outcome may be in the form of product innovation (e.g. sales from innovative products) or process innovation (e.g. productivity gain from innovative process) (Goldstein and Drucker 2006). The distinction between innovation input and innovation outcome indicators is clearly depicted in block 3 of the framework in Fig. 4.

While many studies continue to use innovative product sales or proportion of new products as indicators of firms’ innovative outcome, there are recommendations to refine innovative outcome modes and indicators. In a recent study, Evangelista and Vezzani (2010) expanded Eurostat’s (2012) definition of innovation outcome from two to four modes to examine economic impact: (a) pure product innovation, (b) pure process innovation, (c) organisation innovation and (d) mix of product, process and organisational innovations. Penfield et al. (2014) further articulated the need to distinguish among output, outcome and impact.

The fourth block of the framework comprises studies that use economic indicators such as employment, income, output and GDP growth, as summarised in Table 6.

The impact studies in this category generally applied the input-output modelling or Keynesian multiplier approach. In UK, Harris’ (1997) study of the University of Portsmouth with the input-output approach estimated that the direct expenditure by the university sector resulted in an output multiplier effect of 1.24 to 1.73. Glasson’s (2003) input-output model found an output multiplier of between 0.70 and 1.12 for Sunderland University in Northeast England.

Using Keynesian multiplier approach, Brownrigg (1973) arrived at income multiplier between 1.45 and 1.80 based on a case study of the University of Stirling, while Huggins and Cooke (1997) found the corresponding multiplier at 1.46 to 1.52 at the Cardiff University. In the USA, Felsenstein (1996) applied econometric model on the input-output model to quantify the contribution of the Northwestern University to the Chicago metropolitan area at an output multiplier of 3.1 and employment multiplier of 1.55 in 1993. In a more recent study, Roessner et al. (2013) estimated the impact of university licensing on the whole US economy with a range of hypothetical licensing royalty rates at 2, 5 and 10 %, based on AUTM data (1996–2010) and product substitution effect adjustment.

Despite the popularity of the input-output and Keynesian multiplier approaches, these methodologies have raised some concerns. First, these techniques are restricted to estimating the backward linkages arising only from university expenditures and investment. The dynamic impact of university knowledge production and other forms of output on the business community are not captured (Martin 1998). Second, there is a lack of counterfactual hypotheses for comparison of impact with and without the university’s presence, giving rise to wide variation in results (Goldstein and Drucker 2006, p. 24). Third, the input-output model assumes that product sales arising from university/PRI-licensed IPs are all contained within the local economy to contribute to GDP. This assumption becomes invalid when a significant proportion of IPs are commercialised overseas due to the small domestic market size. Fourth, the input-output model treats university/PRI licensing and research income as expenditures that can have direct and indirect effect on the output and employment of other industries within the local economy. However, in a small country with an open economy, it is unlikely that it has the capability and capacity to provide all the equipment, materials and talent required to conduct R&D.

We understand that different indicators are useful and relevant for tracking the effects of intermediate output or outcome at the different stages of the long journey of research, innovation and enterprise (RIE) value-added activities involving multiple stakeholders from science produced by universities/PRIs through the innovation outcome generated by the industry/firms to economic development:

Given that RIE is a complex and iterative value chain with multiple feedback loops, it is apparent that no single indicator can furnish a complete picture of public-funded research and innovation impact. We therefore propose an integrated Impact Assessment Framework for universities/PRIs as a starting point to manage their value chain. The framework provides the policy makers and scholars the flexibility to define and refine indicators at different stages of the RIE journey while keeping a holistic view of the entire value chain, to facilitate periodic monitoring, analytics and adaptation at micro-, meso- and macro-levels. In particular, the framework includes a relatively new indicator, RICV, to estimate innovation outcome at firm or industry level, as well as economic impact at a regional or national level.