Research Diversity and Invention

In the context of their own original research about the challenges in effectively using knowledge diversity within research teams to create new products, Pollok et al. (2021) provide an integrative discussion of key contributions in the large, well-developed management science literature that finds that diversity benefits innovation because it allows taking advantage of more combinatorial possibilities to discover the right combination for an invention.

Sect. 2 develops the definition of uncertainty and establishes the result that numerous diverse research organizations increase the likelihood of invention by using the description in Scott (2016) of the invention sample space—the set of possibilities for the combinations of essential elements of knowledge in the creative process of finding the right combination for an invention. Scott (2016) used the description to set out the steps in creative individuals’ process of discovering an invention, as individual inventors or within research organizations, and to describe various corporate strategies for sharing information. In this paper, the description is used to define a particular type of uncertainty and then to show how the uncertainty can be resolved by having numerous diverse research organizations. Additionally, this paper develops the potential implication for merger policy of the new ideas that are developed in Sect. 2.

As described by Usher (1929, p. 11), “Invention finds its distinctive feature in the constructive assimilation of preexisting elements into new syntheses, new patterns, or new configurations of behavior.” Invention combines diverse ideas. Weitzman (1998, pp. 334–336) has several noteworthy statements—including Usher’s classic statement—from writers who have explained that the creative process of invention is a process of combining ideas and, moreover, combining quite diverse ideas. The mathematician Poincaré observed in 1908, as quoted by Weitzman (1998, p. 335): “To create consists precisely in not making useless combinations and in making those which are useful and which are only a small minority. Invention is discernment, choices…. Among chosen combinations the most fertile will often be those formed of elements drawn from domains which are far apart.”

Pollok et al. (2021) review the management literature that develops the insight that too much diversity in the points of view that inform an organization’s research process is counterproductive because the research will lose focus and be difficult to manage. They study knowledge diversity’s positive and negative effects on team creativity and “… argue that while diversity creates combinatorial opportunities that increase the likelihood of novel creative output, it also impedes team coordination making convergent refinement towards useful solutions less effective” (Pollok et al., 2021, p. 2).

The increasing costs for working with a larger set of knowledge elements is an example, in the context of a research organization, of the U-shaped cost curves shown in microeconomics textbooks. The organization’s desire to encompass more knowledge elements to increase its likelihood of discovering an invention reflects an “economy of scale” in invention (along with other sources of economies of scale such as the large, expensive equipment that may be needed for some kinds of research). In the textbook case, the U-shaped unit cost curves turn upward in the short run as more variable factors of production are used with the fixed factors of plant and equipment; and even in the long run when all factors of production are variable and plant and equipment can be expanded, the long-run unit cost curves will be U-shaped if there are diseconomies of scale. The classical reason for diseconomies of scale is managerial diseconomies, and that is what the organizational science literature is observing for research organizations that face limitations on the amount of diversity in their knowledge sets that is manageable. One could think of the research organization’s knowledge set as an asset, just as the specialized equipment that its researchers use, and in the long run the research organization’s knowledge set can be expanded, adding more elements of knowledge to the knowledge set that is used with its human and physical capital. However, the difficulties of managing the organization increase, the organization incurs costs of dealing with those difficulties; and eventually the cost of a unit of research output increases with further expansion of the knowledge set.

For examples of the requisite t elements of knowledge for each of several prominent inventions, see Scott (2016, Table 1, pp. 413–414).

The summation of p(f) from f = 0 to t equals 1. The proportions are not probabilities: The set of possible outcomes has been described; but a numerical value for probability has not been associated with each potential outcome. If for example all potential outcomes were equally likely, then the proportions would be probabilities; but that is not the case.

Restricting the portion of the universe of knowledge from which a research team chooses its s elements of knowledge for study can—if the portion’s set of knowledge elements includes the requisite t elements—increase the proportion, of all the ways that the s elements could be chosen, for which the chosen set of s elements includes the necessary t elements. See the discussion of the “exclusion step” in Scott (2016, pp. 421–425).

If a research team works with s elements of knowledge from which could be chosen a combination to create an invention, the total number of possible combinations in distinct groups with one or more of the elements is \({2}^{s}-1\). See the explanation and discussion in Scott (2016, n. 8, p. 427) and the references there. Adding one element of knowledge to the set that is used by the researchers essentially doubles the number of possible combinations. The ratio of the combinations of s + 1 to combinations of s is \(({2}^{s+1}-1)/\left({2}^{s}-1\right)=(2-\left(1/{2}^{s}\right))/(1-(1/{2}^{s}))\), which approaches 2 as s becomes large.

For our example with n = 1,000,000, s = 1,000, and t = 3, if—to provide an illustrative example—v = 61, then computing the change in the change using the formula given by Eq. (4) yields the answer 53/(8,333,308,333,350,000) = 6.36001908004452… x 10^− 15. Although we need Eq. (4) to prove that the proportion P_t increases at an increasing rate as s is increased, we can compute the change in the changes more directly. One gets the same answer if Eq. (1) is used to compute the values for P_t for the cases when s = 1060, s = 1061, and s = 1062 (with n = 1,000,000 and t = 3) and then computes directly, from those values for P_t, the change in the two successive changes as s is increased by 1 from 1060 to 1061 and then again from 1061 to 1062. Also, the same answer is found using Eq. (2) to compute the two successive values for P_t and then computing the changes and the change in the changes. Further, the same answer is found if Eq. (3) is used to compute each successive change and then the change in the changes is computed.

The assumption here is independence across research organizations with regard to whether a research organization has in its knowledge set the t elements of knowledge essential for the invention pursued. However, even if there were some positive correlation across organizations in the presence of the t elements, the result here remains: A larger number of organizations implies a larger probability that at least one organization has the essential t elements in its knowledge set. Moreover, a larger number of organizations implies a larger probability that somewhat similar (differentiated) inventions can emerge as the successful invention that results when the numerous diverse organizations are pursuing “the invention” and several find the successful combination of knowledge elements, differentiating their versions sufficiently to allow obtaining intellectual property or at least avoid rival organizations blocking the marketing of their versions. For example, as documented in Danziger and Scott (2022), there have been many competing versions of drug-eluting coronary stents, with the variations in them sufficient to allow the differentiated and patented products to be marketed. Danziger and Scott also explain why theoretically the competition that involves similar products might not have a detrimental effect on R&D investment.

Thinking in terms of the “demand” for and the “supply” of inventions is helpful for understanding where the findings here fit within the industrial organization literature. For a cost-reduction invention, the demand comes from an enterprise (or from an entrepreneur) that seeks a lower-cost way of producing some product or service. The strength of that demand will depend on the market structure of that product or service—as has been developed in the large literature that was spawned by Arrow’s (1962) seminal article. The supply will be a function of the elements/diversity argument that is developed in this paper, economies of scale, and the difficulties of managing a large research organization. There is also the “vertical integration” issue of in-house versus stand-alone research, which then raises the familiar issues of appropriability, asymmetric information, and financing, among others. A similar demand and supply framework could apply to an invention that creates a differentiated substitute for an existing product.

With these iconic examples, individual researchers and research organizations shared information, “borrowing” ideas from others. In addition to the discussion in Pollok et al. (2021) about the open sharing of information among researchers, see Meyer (2015, pp. 214–215) who provides, with citations of numerous important contributions, an excellent window on the very large literature about the benefits, for invention and innovation, of freely sharing information. Meyer provides references to the works of many scholars (for example, von Hippel (2005) with his focus on user (consumer) led innovation) who have made contributions to the concept and understanding of open innovation that occurs when individuals, firms, and organizations more generally find sharing ideas to be advantageous. In the context of the sample space for invention, Scott (2016) discusses the sharing of information as a corporate strategy.

For example, the inventors of the drug-eluting coronary stent benefited from a vast amount of research from diverse research organizations about the causes of narrowing of coronary arteries, about the methods of treatment, and about anticancer drugs that prevent unwanted proliferation of tissue cells. Observing the two NIH inventors brainstorming the idea for the drug-eluting coronary stent, demonstrating proof of concept, and obtaining a patent, we need to remember the numerous diverse research organizations that preceded and made possible the invention and effectively reduced n to a much more manageable \(\widehat{n}\) by the time that we observe the invention’s discovery. See Nijhara et al. (2005) and Danziger and Scott (2022).

Monopoly is unlikely to replicate the diversity that free entry of diverse research organizations would provide because of the difficulty of managing a large knowledge set (large s). Moreover, the monopolist would not have an incentive to incur the costs of ensuring diverse outcomes for research trials to increase the likelihood of a dominant innovation, because the monopolist gains the same expected benefit regardless of the number of successful trials yielding substitutable innovations. Substitutable products that are commercialized because of the monopolist’s research trials will not create rent-eroding price competition in the post-innovation market (Scott, 1991, p. 139).

The theoretical and empirical evidence that supports the idea that diversity across research organizations increases with their number is now three decades old, and it would be useful to have new research on the subject. The research from three to four decades ago was able to make use of the data that were provided by the now discontinued Federal Trade Commission’s Line of Business Program. Appropriate new data are now available: Three decades after the FTC stopped gathering line of business data, the U.S. National Science Foundation introduced the Business R&D and Innovation Survey (the successor to the Survey of Industrial Research and Development), which provides a source of high-quality R&D data that are segmented by type of business that can be combined with the Census Bureau’s Longitudinal Establishment Data file (see Scott, 2014).

Gilbert (2020), which will be discussed subsequently, reviews U.S. and European Commission (EC) merger cases that address innovation competition; and Martin (2008) provides an insightful and authoritative overview, comparison, and contrast of U.S. competition policy with policies in the European Union.

In the United States, the key federal laws that could be used to protect diversity in the innovation process are the Sherman Antitrust Act (26 Stat. 209, 15 U.S.C. 1 and following), the Clayton Antitrust Act (Public Law 63–212, 38 Stat. 730, 15 U.S.C. 12 and following), and the Federal Trade Commission Act (Public Law 63–203, 38 Stat. 717, 15 U.S.C. 41 and following). The key U.S. federal enforcement agencies are the Antitrust Division of the U.S. Department of Justice (DOJ) and the Federal Trade Commission (FTC). The U.S. federal courts interpret the law in the context of the cases that are filed by the DOJ and the FTC, as well as by private parties and by state Attorneys General.

A reader of the earlier version of this paper observed the tension between my endorsement of Baker’s article and my subsequent explanation of how antitrust policy has ignored, or had inadequate tools to address, the need to consider the importance of numerous diverse research competitors in a market. The resolution of the tension is that Baker’s view is based on an understanding of the literature as it was, while in this article I advance a new dynamic-efficiency-based reason for change in both the economics and the law.

Such a change in policy would be consistent with the competition and innovation policies that are advocated by Aghion et al., (2021) to promote what they term “inclusive growth,” although the policy change proposed here is a narrow, focused one that is based on the idea that was developed above. Aghion et al. (2021) consider a much broader range of policies, including some akin to those in Federal Trade Commission (October 2021, p. 21) as will be discussed below. They also explicitly place their analysis in the worldwide context of emerging markets and developing economies as well as advanced economies.

Providing an overview of the literature about competition and innovation that is used in competition policy, Hovenkamp (2016, p. 1) observes, “A broad consensus today is that the market structure/innovation curve is a lopsided, inverted “U.”. . . Neither monopoly nor atomistic competition is especially conducive to innovation. Rather, most innovation occurs in moderately competitive, product differentiated markets.” The broad consensus has a firm foundation in the literature. As Scherer (1970, p. 378) surmised early in the development of the literature: “What is needed for rapid technical progress is a subtle blend of competition and monopoly, with more emphasis in general on the former than the latter, and with the role of monopolistic elements diminishing when rich technological opportunities exist.” The rich technological opportunities allow sufficient appropriation of returns to support the investments of firms in more competitive markets. For an important review and perspective about the early theory and evidence of the inverted-U relation between market structure and innovation, see Scherer & Ross (1990, pp. 630–651). Baker (2007, p. 584) discusses papers subsequent to the Scherer and Ross review that “appear to have resurrected the “inverted U” result” but “do not control satisfactorily for differences across industries in the extent and rate of growth of technological opportunity and in the conditions of appropriability.”

Williamson (1968) provides the landmark examination of the increased economic performance from efficiency-enhancing concentration of a market’s sellers despite an increase in their market power: the ability to control price.

The enforcement agencies consider a moderately concentrated market to have a Herfindahl-Hirschman Index (HHI) between 1500 and 2500 (U.S. Department of Justice and Federal Trade Commission, 2010, p. 19), which corresponds to a market with between 6.67 and 4 equal-sized firms. Some substantial mergers in moderately concentrated markets could in theory be challenged, but the mergers that are challenged by the agencies typically are in markets that are highly concentrated, with HHI greater than 2500—often considerably so.

The evidence in Scott (1984) and the theory and evidence in Scott and Scott (2014) suggest that the sightings of the inverted-U have been the result of insufficient controls for the many forces that affect R&D investment.

Kwoka (2017), who examines U.S. policies, and Affeldt et al., (2021), who examine European Union policies, conclude that antitrust enforcement has been too lenient because too much weight has been placed on the possibilities for efficiency gains as compared to the effects on market power from lost competition.

Gilbert’s (2020) careful review of U.S. and EC merger cases that have addressed concern about innovation competition supports the view that protecting large numbers of diverse research rivals is not a focus of merger policy. As was discussed above, Baker (2007, p. 592) observed that the merger challenges that entail a concern about innovation occur when there are just a few likely innovators—and even then, only when offsetting innovative efficiencies are not anticipated. The enforcement actions—whether to block a merger, require divestiture of assets, or require licensing of technology—occur when few research rivals remain in the market. Gilbert (2020, pp. 163–165) does provide what might appear to be an exception—the EC’s challenge to a merger of the agrochemical businesses of Dow and DuPont. Conforming with the norm of addressing loss of innovation competition only when a few innovation rivals remain, when it challenged the merger, the U.S. DOJ did not require the divestiture of any R&D assets. However, the EC did require DuPont to divest R&D assets for agricultural pesticides, applying “… unilateral effects theory for early-stage R&D incentives in an industry with more than a few potential innovators” (Gilbert, 2020, p. 164). Although the case is novel, Gilbert’s detailed discussion explains that although there are many research organizations doing R&D for agricultural pesticides, the EC specifically restricted the set of potential innovators in various ways that left just a few in the relevant “innovation spaces.” Thus, even this one novel case conforms to the enforcement agencies’ concerns (about a merger lessening innovation competition) only when a few research competitors remain.

The FTC even voted to withdraw its 2015 statement about its enforcement principles as a part of its decision to take a more expansive approach to its enforcement efforts under Sect. 5 of the FTC Act (Federal Trade Commission, July 1, 2021). The withdrawn 2015 statement enumerated “the Commission will be guided by the public policy underlying the antitrust laws, namely, the promotion of consumer welfare; the act or practice will be evaluated under a framework similar to the rule of reason, that is, an act or practice challenged by the Commission must cause, or be likely to cause, harm to competition or the competitive process, taking into account any associated cognizable efficiencies and business justifications; and the Commission is less likely to challenge an act or practice as an unfair method of competition on a standalone basis if enforcement of the Sherman or Clayton Act is sufficient to address the competitive harm arising from the act or practice.” (Federal Trade Commission, 2015). For the meaning of the goal of promoting “consumer welfare,” see Martin (2008, pp. 46–49).

Thinking about the entire spectrum of proposals to use antitrust policy to address more than maintaining competition in markets, White (2021, p. 14) observes, “The focus of antitrust is narrow: maintaining competition, with the goal of enhancing the efficiency and innovativeness of markets in ways that ultimately benefit consumers. To widen the goals of antitrust—as has been advocated by some critics of current antitrust policies …—so as to encompass goals such as expanded employment, a cleaner environment, improved income distribution, community development, etc., would be a mistake. It would dilute and distract antitrust enforcement, since additional tradeoffs between efficiencies and these other goals would need to be considered—and additional data and modeling would be needed so as to measure/quantify these tradeoffs. The current tasks of antitrust enforcers are already difficult; to add dimensions that they would have to evaluate and for which they would have to acquire expertise and determine tradeoffs could make their tasks near-impossible. Instead, if antitrust enforcement is perceived as contravening other goals … then there should be other policies that are brought to bear to provide appropriate remedies.”

The FTC’s intention to address equity is made explicit in the title to the section quoted: “Objective 2.4: Advance racial equity, and all forms of equity, and support underserved and marginalized communities through the FTC’s competitive mission.” (Federal Trade Commission, October 2021, p. 21).

The new policy proposed would be consistent with Gilbert’s (2020, p. 235) call for competition policy to become more “innovation-centric” while not abandoning its traditional “price-centric” orientation.

Scale economies in R&D could arise not only from the “traditional” reasons—research labs are expensive, so are experimental nuclear reactors, etc.—but also (building on the “elements” structure of inventions) from the reason that some companies may be better at managing larger numbers of the elements that need to be combined into an invention. This is wholly parallel to the idea that some senior executives are better at managing larger production processes, larger (deeper) vertical processes, and/or larger (wider) scope processes.

Note that the implementation of the change in policy would not necessarily require antitrust enforcers to evaluate the diversity in individual cases: Perhaps enforcers could favor a merger if the scientists and engineers in the two firms came from different disciplines or different graduate programs, work in different cities, or attend different research conferences, or favor the union of the research organizations if they each pursued different theoretical approaches in their research programs. The reason is that because of the increasing costs that limit the number of elements of knowledge that can be effectively managed within a research organization, the merger of such diverse firms would be likely to eliminate their diversity. Arguably a good approach would be for the enforcers to use our understanding that the diversity of knowledge sets is positively related to the number of research organizations and, on the diversity dimension, consider just whether the merger reduces the number of diverse competitors—although, as was noted above, more research is needed on that subject. Merging firms might argue that they planned after the merger to continue their individual research streams and work with the elements in their individual knowledge sets; but the argument that has been developed in this paper is that the increasing costs associated with expanding an organization’s knowledge set would be likely to cause the merging firms to abandon the plans to maintain the pre-merger diversity of their research.

With reference to note 11, observe that whether multiple firms that invest in research discover “the invention,” or instead just one firm finds it, many firms may be able to commercialize it by finding ways to differentiate their version of the invention. Alternatively it is possible that product market competition will instead require multiple inventions, with each distinguished as a different combination of elements of knowledge but with each of the differentiated products interchangeable in use. Whatever the case, the central finding remains that blocking mergers to preserve independent centers of research will increase the likelihood of successful invention because it will increase the chances that one or more research organizations will be working with knowledge sets that include the requisite elements for invention.

The history of the antitrust law regarding resale price maintenance challenged under the Sherman Act illustrates the slowness of the evolution of antitrust law in response to new understanding about the economics of the behavior involved. Close to a century passed from the U.S. Supreme Court’s precedent-setting opinion in the 1911 Dr. Miles case (220 U.S. 373) until it was overturned in the Supreme Court’s 2007 decision in the Leegin case (127 S. Ct. 2705).

The findings of Bloom et al. (2020) about a decline in the productivity of scientific research are consistent with Gordon’s (2016) historical analysis of the slowdown in economic growth for the U.S. economy and with Jones’s (2009) observations about inventors’ difficulties of reaching the frontiers of knowledge as knowledge accumulates. See also the evidence and discussion in Link and Scott (2021) about the increase in the resources needed to achieve gains in knowledge.