The most damaging of the several myths surrounding innovation is that it is all about ideas. Managers and researchers are both led into counter-productive behaviors as a result of this myth. This is not to discount the importance of ideas in innovation – every innovation starts with a great idea. What we will argue, however, is that generating great ideas is actually the easiest part of innovation. Improving innovation comes not through generating more ideas, but through implementing existing ideas more effectively.
We advocate a shift towards viewing innovation as requiring the managing of ideas. This is based on the important distinction between invention and innovation. We define invention as the creation of a novel idea, while innovation is executing a new idea to create value. Conflating the two concepts leads to problems for researchers and managers alike. Indeed, poor theory on the dynamics of innovation is probably leading to detrimental practices in organizations. Business schools that teach these theories bear some responsibility for putting these stylized abstractions of organizations into practice.
Innovation is often thought of as the product of process (Van de Ven et al., 1999). This does not mean that the outcome of any particular innovation initiative can be predicted in advance, or managed to a guaranteed outcome. Rather, it means that innovation efforts are more successful when they are managed systematically (Anthony et al., 2008). This seems like an obvious statement for anyone who has observed innovation within a firm, but surprisingly little research has been carried out on how best to understand this process.
In the organizational behavior literature, it has been acknowledged that innovation is not a singular event, and there have been calls for a clearer understanding of different phases of innovation (West et al., 2006). Behaviors and skills that might be productive for idea generation may be superfluous in the implementation of the idea. For example, creativity may be a major factor for the development of a solution, but less significant for working through the technical details of putting the solutions to work (West, 2002).
One of the frameworks used to conceptualize innovation is the innovation value chain (Hansen and Birkinshaw, 2007; Roper et al., 2008). This model envisages three steps in innovation: idea generation, idea selection and testing, and idea diffusion. Hansen and Birkinshaw contend that organizations cannot successfully innovate unless they are proficient in all three steps. While their model was written as a Harvard Business Review paper, intended for practitioners, Roper et al. have validated it as a robust model for examining innovation. Although the general model is sound, more work needs to be done to develop diagnostic questions around performance in different parts of the value chain. Hansen and Birkinshaw report none of the usual measures of construct validity in their survey instrument.
This value chain model has been extended to five steps in two recent publications discussing public sector innovation (Eggers and Singh, 2009; Management Advisory Committee, 2010). These models break the second step into two discrete items, and add a step for embedding the new ideas internally prior to diffusion. By using the innovation value chain questionnaire, organizations can identify the part of the chain in which they are weakest. The tool is designed to be used iteratively, so that as soon as the problems in one area are corrected, the organization can move on to tackle the problems in the next.
We understand that our colleagues in the academic community may not share our desire to research innovation as a process, with the objective of producing a stage model. The staged and linear model of science commercialization fell out of favor many years ago (Dodgson and Gann, 2010). The problem is that the business community needs valid models for managing innovation. When we are asked by a manager for some guidelines for setting up an innovation program, we can find little of value in the academic journals. Instead, we invariably recommend articles in the Harvard Business Review and the occasional pop management book. While some of these are based upon empirical research, many are not. There is an urgent need to research innovation as a management activity. This has been a valuable direction in the strategy literature, and we now know much more about productive strategy practices (Whittington, 2003; Jarzabkowski, 2005). Researching innovation management practices to identify successful and unsuccessful activities is just as important.
The problem for management
For the past year, we have been using the innovation value chain as part of the assessment in our MBA classes. In that time, over 70 people have used the questionnaire included in the Hansen and Birkinshaw paper to assess innovation within their own organizations. These people come from the full range of organizations – Fortune 500 multinationals, start-ups, governmental departments, educational institutions and research centres. They also cover a wide range of industries. While the assignment requires the individual to assess their own organization, many people have actually distributed the questionnaire to other members of their organizations. The result is that we have information from more than 300 people in 65 organizations. Of these 65 organizations, only three (4.5%) identify themselves as ideas poor. The remaining 62 are about equally split between having problems with idea selection and idea diffusion.
This illustrates the first problem with conflating invention and innovation: thinking of innovation as consisting only of generating ideas means that more than 95% of organizations will not even think about the parts of innovation in which they are weakest. It follows that organizations viewing innovation as simply coming up with ideas will invest in the wrong part of the process. They will sink time and resources into improving idea generation when this may actually be their strongest area.
The impact of this is demonstrated by our experiences with one of our research partners. This is a manufacturing organization employing about 10,000 people. It approached us for help in embedding an innovation culture. In a relatively stable market, the organization’s managers had never viewed themselves as innovative. However, innovation was one of the organization’s five core values and upper management had decided that it was finally time to convert aspiration to reality.
The first thing that they did was buy software designed to improve idea capturing and generation. They also instituted an informal policy that ‘there are no bad ideas’ in order to encourage people to come forward with their ideas. What was quickly discovered was that the firm was full of ideas. The software was soon full with over 800 new ideas.
This led to two problems. The first was that there was no mechanism in place for prioritizing or selecting ideas. Consequently, when we reviewed the program about a year after it started, none of these 800 ideas had been killed. The second problem was that there was no provision for attaching budget to ideas that were acknowledged as good. At the time of our review, about 15 ideas had been exploited. When we used the innovation value chain to evaluate their innovation, many interviews with staff revealed that the weakest part of innovation was the selection and execution step. The organization was actually pretty good at generating ideas, and reasonably skilled at idea diffusion as well.
The experience of this organization is not uncommon. Nearly all the resources available to help firms improve their innovation are based around generating ideas. There are countless consultants and trainers available to help produce ideas. Brainstorming, gamestorming, design thinking, crowdsourcing, and the TRIZ (Teoriya Resheniya Izobretatelskikh Zadatch) theory of inventive problem solving are just some of the techniques widely available for improving idea generation. These all address the part of the innovation process that creates problems for less than 5% of the organizations we have examined. Why is this? The first reason is that when innovation is not managed as a process, the natural response is to focus on idea generation because this is all that people think of. A second reason is that idea creation is easily measured. This makes it the most attractive part for consultants because it is the easiest area in which to demonstrate results.
The problem for research
This ease of quantification is also the primary problem that the focus on idea generation poses for research. As in the management of innovation, researching innovation is prone to conflating idea generation with the process of innovation as a whole. In a review of the behavioral psychology literature on innovation, West (2002) suggests that the emphasis on idea generation has come at the expense of studying implementation:
…such ideas for new and improved ways of doing things are ten a penny. Implementing them is what counts and that is most difficult. And therefore, what I have argued here, that is what we should be studying. (West, 2002, p.421)
We give two examples of the research bias towards examining ideas rather than what makes these ideas work. The first example is demonstrated by the almost complete reliance on patent data in quantitative studies of innovation. The second example is the relatively limited discussion of processes in the literature on ‘open innovation’.
From a researcher’s standpoint, patents are an attractive data source. Patent data are easy to access, and come in very large sets with a great deal of complementary data available for each patent. Consequently, the statistical power of regression analyses based on patent data makes the generation of statistically significant results possible, which enables researchers to make strong statements (McCloskey, 1998).
The major flaw in the use of patent data is that they do not measure innovation. Rosenberg made this point in a series of books (Rosenberg, 1976, 1982, 1994) where he argues that idea generation must not be considered separately from issues of development and diffusion. Concepts such as the innovation value chain certainly have their roots here and in the chain-link model developed by Kline and Rosenberg (1986), which conceptualizes innovation management as a non-linear multi-step process. Patents pick up only a part of this:
The use of patent statistics to measure innovation rather than simply to count patented inventions says much more about the perceptions of the users than the statistics say about innovation. (Macdonald & Lefang, 2003, p.134)
Others who have noted this weakness in using patent data as a measure of innovation include Smith (2005), who, in an otherwise positive discussion of patents, points out:
Patents also of course have weaknesses, the most notable of which is that they are an indicator of invention rather than innovation; they mark the emergence of a new technical principle, not a commercial innovation. Many patents refer to inventions that are intrinsically of little technological or economic significance. (Smith, 2005, p.160)
Consequently, research based on patent counts as a measure of innovation misses the critical steps required to successfully derive economic benefit from invention:
Technologies acquire economic value when they are taken to market with an effective business model. When research discoveries are driven by scientific inquiry and are not connected to any business purpose, the commercial value of the resulting discoveries will be serendipitous and unforeseeable. (Chesbrough, 2006, p.161)
Even worse, patents provide only a partial measure of economically important invention. Patents are poor at picking up innovation in services, business processes and business models (Chesbrough, 2006). Patents are poor at picking up innovation in small and medium enterprises (Macdonald, 2004).
It is obvious that the patent indicator misses many non-patented inventions and innovation. Some types of technology are not patentable, and, in some cases, it is still being debated whether certain items (eg, new business formulae on the internet) can be patented. … The above implies that, when using patents as an innovation indicator, we are likely to make four types of systematic mistake. First, we underestimate innovation in low technological opportunity sectors. Second, we over-estimate innovative activity among firms that collaborate on R&D. Third, we underestimate the rate of small firms that innovate. And finally, we overestimate the innovation intensity of small-sized patent holders. (Kleinknecht et al., 2002, pp.112–13)
Kleinknecht et al. (2002) conclude by discussing alternative measures of innovation that have fewer systemic problems than patents. Patents do not measure innovation and do little to help us understand processes.
The collaborative aspect of innovation is not new, but since the publication of Chesbrough’s (2006) influential book on open innovation, there has been a renewed interest in how innovation works in partnerships among organizations. In Chesbrough’s model of open innovation, firms can be open with respect to sharing or selling ideas, technologies or businesses (Dahlander and Gann, 2010). However, there has been a tendency to consider openness with respect to knowledge and ideas alone, as if ideas are the most challenging part of the innovation process. Data from the community innovation survey are convenient for researchers (e.g. Laursen and Salter, 2006) because they help identify customers, suppliers and others as sources of innovation, but the survey tells us little about what is being sourced from outside the organization and how this relationship actually works. Equally, the inward or outward licensing of patents tells us that information is being shared among organizations, but tells us nothing of the process of turning this information into valuable products and services.
Improving research on innovation
In part, the difficulties in managing innovation reflect these difficulties in researching innovation. If innovation is not conceptualized as a process by academics, where will managers learn to think of it as a process? One stream of relevant literature considers innovation as a dynamic capability where consistent routines enable firms to develop better goods, services and production methods (Eisenhardt and Martin, 2000). This implies that rather than being ad hoc and unmanageable, there are some aspects of the innovation process that are identifiable and have relatively clear stages.
Conceptualizing innovation as a dynamic capability implies commonalities in innovation processes and will assist in distinguishing practices that help or hinder innovation outcomes. In order to improve both the research on, and the management of, innovation, the following research questions should be more widely addressed:
1. What are the stages in innovation?
Such tools as the innovation value chain have proven useful in practice, but they are not fully supported by theory, nor have they been fully researched. As the literature on the topic shows, it is possible to visualize innovation as a three-step process or as one with five steps. This is actually an empirical question that needs to be researched in more detail. And do the steps in the innovation process differ among sectors? Are there situations in which it is better to visualize the innovation process in a particular way?
2. Are organizations generally better at some parts of innovation than others?
Our classroom data suggest that, in general, organizations are not ideas poor. This also requires better empirical documentation. If this is indeed broadly the case, then we need to develop better understanding of which parts of the process are generally well executed by organizations.
3. What tools can be used to manage effectively the different parts of innovation?
The vast majority of tools available to innovation managers are oriented towards creating ideas. We need more tools that address the parts of innovation that are less well managed. Innovation jams, a collaborative innovation activity pioneered by IBM (Bjelland and Wood, 2008; Dodgson and Gann, 2010), have proven to be useful not just for generating ideas, but also for selecting them. The stage-gate process can be used to manage both the selection and the execution steps (Cooper, 2001). What other tools are available? If there are no suitable tools, can we develop some?
4. What metrics can measure innovation more accurately?
This is probably the most critical research question. The simple fact of the matter is that patents do not measure innovation. Consequently, normative conclusions based on patent data must be viewed with suspicion. There has been some progress in measuring innovation at the organizational level (see, for example, Anthony et al. 2008; Management Advisory Committee, 2010), taking a processual approach to measuring innovation. We need to bring these sorts of measures into wider use, and we also need to develop similar metrics to use at the sectoral and regional levels.
Conclusions
The conflation of idea generation with innovation is dangerous to both research and practice. If researchers are to provide better normative guidance to practitioners, they must address this problem. This starts at the conceptual level: if the management of innovation is ever to become a dynamic capability, innovation must be viewed as a process rather than as an event.
The problem with patents is that they measure only information. This makes them an inappropriate proxy for innovation. Consequently, research on innovation outcomes must no longer be based on patent data. The reliance on easy access to large patent data sets leads to faulty theory building and poor normative prescriptions. Innovation researchers need to develop better measures of the innovation process in order to improve the relevance of their work.
This is an important issue for academics in particular. Innovation research based on patent data does not provide useful information to practitioners. If we are to have an impact on the effective management of innovation, this problem must be addressed. If it is not, managers will be left with only fads and charlatans to provide them with guidance.