SciELO - Scientific Electronic Library Online

 
vol.6 issue3What Holds us Together?: Analyzing Biotech Field FormationTo Harmonise or Not to Harmonise?: The Case of Cross-national Biotechnology Governance in Southern Africa author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

Share


Journal of technology management & innovation

On-line version ISSN 0718-2724

Journal of Technology Management & Innovation vol.6 no.3 Santiago Oct. 2011

http://dx.doi.org/10.4067/S0718-27242011000300002 

Journal of Technology Management & Innovatio
J. Technol. Manag. Innov. 2011,Volume 6, Issue 3

ARTICLES

Measuring Creativity for Innovation Management

 

David H. Cropley1, James C. Kaufman2, Arthur J. Cropley3

1 Associate Professor, Defense and Systems Institute, University of South Australia, Mawson Lakes Campus, Mawson Lakes, SA 5095 AUSTRALIA, Phone: +618 8302 3301. Email: david.cropley@unisa.edu.au

2 Associate Professor James C. Kaufman, Learning Research Institute, California State University, San Bernardino, 5500 University Parkway, San Bernardino, CA 92407 USA, Phone: +909 537 5000. Email: jkaufman@csusb.edu

3 Emeritus Professor AArthur J. Cropley, University of Hamburg, C/- 3/120 South Terrace, Adelaide, SA 5000 AUSTRALIA, Phone: +618 8212 2137 Email: ajcropley@gmail.com


Abstract

Identifying the extent and nature of the creativity of new products is a key for innovation management. The revised Creative Solution Diagnosis Scale (CSDS) is a 27-item scale based on a theoretical model of functional creativity, consisting of five main criteria: Relevance & Effectiveness, Problematization, Propulsion, Elegance and Genesis. The CSDS offers potential for differentiated assessments of product creativity as part of the larger process of innovation. Non-expert judges rated a series of mousetrap designs using a 30-item version of the CSDS. Confirmatory factor analysis revealed a simple structure that corresponded closely to the a priori theoretical model of functional creativity. The untrained judges were able to use the scale with a high degree of reliability and internal consistency. The scale offers a tool for managing innovation, especially for stimulating creativity and diagnosing the creativity of products.

Keywords: Innovation; creativity; management; products; measurement; functional creativity; diagnosis.


 

Measuring Creativity for Innovation Management

Bledow et al (2009) defined innovation as the development and intentional introduction into practice of new and useful ideas by individuals, teams, and organizations. The term "value innovation" (e.g., Kim and Mauborgne, 2004; Dillon, Lee and Matheson, 2005) is somewhat more explicit: It focuses on innovation as a process through which organizations find novel and effective ways of serving their current customers and identifying new markets, thus linking innovation to what customers value. This terminology makes it clear that, at the level of organizations, innovation is not just a matter of coming up with a new idea but requires a valuable product, although "product" is not confined to devices or even tangible objects, but covers the full value chain, including marketing, market research, sales, advertising, distribution and customer service. In recent years it has become almost axiomatic that the innovation process is a key one: At the macro level (for instance national innovation policy) it is accepted as vital in meeting the challenges of the early 21st century arising from technological advances, social change, globalization, and now the global financial crisis, while at the meso level of the individual organization innovation is "a key to organizational effectiveness and competitive advantage" (Davis, 2009) and thus ultimately to commercial success and creation of wealth.

Haner (2005) emphasizes that there is an overlap between the processes of innovation and creativity: "Creativity processes and innovation processes are ... different, but they display common characteristics and patterns that allow for joint reflection." Roberts (1988) reinforces this duality: He divided innovation into two stages or phases: Invention and Exploitation. Invention involves the generation of novel ideas, and Exploitation involves the implementation of these ideas in the sense of value innovation. Bledow et al (2009) made a similar distinction, and explicitly identified the first phase (Invention1) with creativity. They saw the complete process of innovation as involving novelty production (in effect, creativity) plus innovation implementation (Exploitation). Looked at in this way, creativity is not identical with innovation but does form an indispensible element of the two-part process just outlined. West (2002) stated the matter quite explicitly: "Creativity is the development of ideas, while innovation ... is the application of ideas."

West also envisaged the joint action of creativity and innovation implementation as sequential: "Innovation then can be defined as encompassing both stages—the development of ideas—creativity; followed by their application (emphasis added)". However, more recent organizational theory does not see the interplay between creativity and innovation implementation as strictly sequential, with creativity always preceding innovation implementation and being completely separate from it. Haner (2005) concluded that "both creativity and innovation processes need to be seen as complex, partly iterative and partly simultaneous efforts". Thus, the two elements of innovation may alternate, or may occur simultaneously. According to Gupta, Smith and Shalley (2006), the synchronous pursuit of both creativity and innovation implementation involves "organizational ambidexterity," whereas sequential processing involves "punctuated equilibrium."

Adams, Bessant and Phelps (2006) highlight the importance of measurement across a range of relevant process categories, including process inputs and knowledge management, for successful innovation. Davila, Epstein and Shelton (2006) stress the need for process measures, including measures of the creative process, to track the quality of ideas, while measures of product performance focus on product ratings in comparison to competitors. Besemer and O'Quin (1999) turned their attention to products, citing Stone-Romero and Stone, (1997), and argued that: "Persons involved in the development and evaluation of products need ways of assessing the creativity of those products. In the business world, for example, it is critical to new product development to have adequate, reliable, and valid means for evaluating the quality, including the creativity, of the products brought to market."

In the field of advertising White and Smith (2001), citing Bell (1992), draw attention to the role that creativity plays, not only in assessing the value of the advertising "product" itself, but also, in some cases, in determining For the sake of simplicity, we will use Roberts's terms.

how advertisers are rewarded by their clients. White and Smith (2001) summarize this by stating that "the judge of the creative product needs an instrument by which to measure creativity". McAdam and Keogh (2004) similarly highlight the role that "supporting measures" play in ensuring effective creativity and innovation, and therefore increased competitiveness. They cite Freel (2000) to link small-medium enterprises and measurement in this context "...while SMEs are disproportionately responsible for near-to-market innovations, they are hindered by lack of creativity and innovation measurement." It is thus clear that the effective management of product innovation begins with a process of managing the production of novelty - that is, managing creativity. In the context of tangible, engineered artifacts, if not products in their broadest sense, effective management of creativity requires a means for measuring product creativity.

Creativity

There is widespread agreement in the literature that creativity requires the ability to produce outcomes that are novel, high quality and appropriate to the task (e.g., Sternberg, Kaufman and Pretz, 2002). Cropley and Cropley (2010a) expressed this concept as the generation of effective novelty.

These outcomes may include products, services, ideas, processes, or procedures (Woodman, Sawyer and Griffin, 1993). Many have argued that creativity drives the broader innovation process of modern economies (Florida, 2002). Christensen (1999) stresses the key role that creativity plays in the innovation process: it is ". about how to find ideas for new products and services that will be unique and valued in their markets". Creativity drives innovation both by underpinning the individual and organizational skills needed to adapt to the pace and nature of change in the modern world (Carnevale, Gainer and Meltzer, 1990), and by acting as a key ingredient in the process of generating new business opportunities, whether in the form of a product, process, system or service.

Puccio and Cabra (2010), in discussing the role that both individual and organizational creativity play in driving innovation, also draw attention to the fact that "innovation comes about as the result of the interaction among people, the processes they engage in, and the environment in which they work" (p. 149). However, it is the end result, the way that these variables interact to form a product, which is critical (Puccio, Murdock and Mance, 2007). Christensen (1999) cautions that "the root cause of many pathologies in product engineering projects is that firms often lack a strategy ... to guide managers in selecting and evaluating projects". Given the importance of creative products to the success of the innovation system, it is therefore axiomatic that we understand how to characterize outcomes that are novel, high quality, and task appropriate (Sternberg, Kaufman and Pretz, 2002). The very nature of product creativity, as a competitive resource in organizations, emphasizes the need for its measurement (Horn and Salvendy, 2009). The creative product is not simply the output of a process of creativity, and the starting point for product innovation; it is the embodiment of that creativity and innovation.

Measuring Product Creativity

There have been extensive studies on how to measure a creative product in its broadest sense. O'Quin and Besemer (1999) describe three common approaches used to measure product creativity: indirect measurement, global judgment and criterion-based measurement. These approaches have been developed both in a domain-general and a domain-specific context. Some of the possible solutions include the use of expert raters (Amabile, 1996), divergent thinking-based scoring of creative products for originality or fluency (Reiter-Palmon et al, 2009), or assessment of a product's historical impact (Simonton, 2009).

Horn and Salvendy (2006) offer a detailed comparison of specific product creativity measurement tools, including rating scales and subjective assessments. The former include Besemer and O'Quin's (1987, 1999) Creative Product Semantic Scale (CPSS) and Reis and Renzulli's (1991) Student Product Assessment Form, while the latter is based on Amabile's (1983, 1996) Consensual Assessment Technique (CAT). Horn and Salvendy (2006) also report that the rating scales have been tested in a variety of domains, including art work, cartoons, chairs, advertisements, scientific and creative writing, audio-visual products and social studies. The CAT has been applied to stories, art, poetry and other aesthetic products. Much of the research has been geared toward evaluating either aesthetic or organizational products.

The assessment of aesthetic work (such as a painting or poem) has been extensively investigated for nearly a century (Baer, Kaufman and Gentile, 2004; Cattell, Glascock and Washburn, 1918; Child and Iwao, 1968). Within industrial/organizational psychology or business, assessing creative products may mean studying group creativity or the performance of teams (e.g., Shalley, 2002).

There are, however, surprisingly few studies aimed at assessing the creativity of products in the sense of tangible, scientific or technological products - that is, engineered artifacts or manufactured consumer goods. Where studies do relate to products, in the sense just described, it is primarily in connection with related concepts, such as "usability" (see, for example, Han et al, 2000). Looking at one such domain (mathematics), Mann (2009) argues that many of the current assessments are time-consuming to score; they also tend to be separate instruments designed to measure the specific domains. As a result, most of the work on mathematical creativity assessment cannot be easily applied to related domains (such as engineering).

Functional creativity

Cropley and Cropley (2005, 2008, 2010b) have sought to address this gap in creativity research by focusing on novel products that serve some useful social purpose, labeling their special quality functional creativity. They argued that novelty seems intuitively to take precedence over usefulness in determining creativity. However, in the practical world of products, processes, systems, and services, the most important aspect of an artifact that excites admiration in the beholder is the product's ability to meet customer needs, i.e. its effectiveness. An automobile, for example, must transport people quickly, economically and comfortably over long distances. If it fails to satisfy requirements like these, then it lacks effectiveness and thus cannot be regarded as creative, no matter how novel it is.

Einstein, however, argued that it is not difficult to find novel solutions to problems that achieve the desired effect: The difficult part is finding solutions that are elegant (see Miller, 1992). Grudin (1990) reinforced this idea when he referred to the grace of great things [emphasis added]. Such solutions not infrequently cause a more or less instantaneous "shock of recognition" when they occur, and provoke a "why didn't I think of that?" reaction. Indeed, an elegant solution may look so simple and obvious—after the fact—that people may underrate its creativity or denigrate it as "banal." In addition, products that are not only useful in the situation for which they were generated but can also be applied in other apparently unrelated situations embody genesis. A product may introduce a new way of conceptualizing an area, for instance, by opening up new approaches to existing problems, or by drawing attention to the existence of previously unnoticed problems.

Cropley and Cropley (2005) classified creative products using the four dimensions listed above. They arranged them in a hierarchy ranging from the "routine" product (characterized by effectiveness alone) at one pole, to the "innovative" product (characterized by effectiveness, novelty, elegance and genesis) at the other, with "original" and "elegant" products between these poles. This classification system is shown in Table 1, where a plus sign means that a property is necessary for this kind of product, a minus sign that it is not. The schematic in Table 1 can also be used to demonstrate the position of pseudo- and quasi-creativity: i.e. novelty without effectiveness. The table shows that as a product moves from routine to innovative, it incorporates all the properties of products at lower levels, but adds something to them.

It is important to stress that routine products should not be dismissed out of hand. They may be very useful: In areas such as engineering, for example, a very large number of products perform important functions that benefit humankind and contribute to the advancement of society (i.e., they are useful), even though they may be devoid of novelty. However, because they lack novelty their creativity is qualitatively different. Changes to these products, rather than representing functional creativity, may instead take the form of replication - that is, minor adaptations to existing ideas (Sternberg et al, 2002) that develop what already exists according to existing lines of thought. It is only when products move beyond repetition and effectiveness, and begin to incorporate novelty, that they enter the realm of creativity.

The hierarchical organization of products shown in Table 1 introduces a further important principle into the discussion: Creativity is not an all-or-nothing quality of a product - there are both levels and kinds of creativity. Creativity is not something that products either have or do not have. Different products can have creativity to greater or lesser degrees, or they can display different kinds of it. Beghetto, Kaufman, and Baxter (in press) give the metaphor of the all-or-nothing viewpoint as envisaging creativity as a simple light switch; in more recent years, most theorists have embraced the view of creativity more as a dimmer switch.

Indicators of creativity

To serve as the basis of an instrument for measuring the creativity of products, and therefore as a tool for facilitating effective product innovation management, the internal and external properties outlined in the previous section would have to be recognizable to observers. The most straightforward way of checking whether people really can recognize creativity when they see it is to ask them. This idea is the basis of the method of consensual assessment (for a summary, see Amabile, 1996). Amabile, along with her colleagues, has developed and refined this approach. Furthermore the Consensual Assessment Technique (CAT) is now well known among creativity researchers (Kaufman, Plucker and Baer, 2008). The method involves asking judges, usually experts in the field to which the product belongs, to rate the creativity of a product.

When people are asked to rate the creativity of products in a global way, novices typically show much lower levels of reliability than experts (Kaufman et al, 2008; Lee, Lee and Young, 2005). Novices' ratings also tend to have a low correlation with expert ratings (Hickey, 2001; Kaufman et al, 2008), although in more accessible domains the correlation rises (Kaufman, Baer and Cole, 2009). Quasi-experts (such as advanced students) often represent a middle group; their ratings often correlate with both novices and experts (Hekkert and van Wieringen, 1996; Plucker et al, 2009). The need for experts arising from this inconsistency among novices highlights a limitation of the Consensual Assessment Technique. It can be expensive (to obtain multiple expert raters), cumbersome (all products must be viewed separately by the experts), and time consuming. What is needed is an approach that can be applied quickly and easily by a range of observers, with a high level of consensus. The present article outlines such an approach.

The functional model of creativity outlined above (Cropley and Cropley, 2005) provides a number of broad properties of products that can be used to describe the level and kind of creativity they possess. The question that arises at this point is that of the observable characteristics of products that reveal the presence of these properties. We refer to such characteristics as "indicators" of creativity. Scales developed by psychologists for rating the creativity of products help to develop a basis for developing relatively concrete indicators of the functional creativity of products. An early example is Taylor's (1975) Creative Product Inventory, which measured the dimensions: generation, reformulation, originality, relevancy,hedonics, complexity, and condensation. More recently, Besemer and O'Quin's (1987) Creative Product Semantic Scale defined the creativity of products in terms of three dimensions: Novelty (the product is original, surprising and germinal), Resolution (the product is valuable, logical, useful, and understandable), and Elaboration and Synthesis (the product is organic, elegant, complex, and well-crafted). In the literature of product innovation, similar indicators are found. Goldenberg and Mazursky (2002) report on research that has found the observable characteristics of creativity in products to include "original, of value, novel, interesting, elegant, unique, surprising."

Criteria such as hedonics or elegance are reminiscent of Jackson and Messick's (1965) distinction between internal criteria such as logic, harmony among the elements of the product, pleasingness, and external criteria (i.e., is it relevant, is it useful?). The indicators suggested by Taylor and Besemer and O'Quin give greater weight to internal criteria such as elegance, complexity or logic. They involve a mixture of pure aesthetic, formalist, and technical properties (see Slater, 2006), whereas they vary in the level of openness or closedness they display.

By contrast, the "propulsion model" (Sternberg, 1999; Sternberg, Kaufman and Pretz 2002, 2003) turns directly to external indicators. According to them, a creative product achieves its external effect by propelling a field. They suggested a number of ways in which this can occur: these include conceptual replication (the known is transferred to a new setting), redefinition (the known is seen in a new way), forward and advanced forward incrementation (the known is extended in various ways), redirection (the known is extended in a new direction), reconstruction (new life is breathed into an approach previously abandoned), and reinitiation (thinking begins at a radically different point from the current one and takes off in a new direction). Savransky (2000) also discussed the processes through which existing knowledge leads to effective novelty in the external world: he argued that inventive solutions to problems always involve changing what already exists, and discerned a variety of ways in which this can occur, including improvement, diagnostics, synthesis, and genesis.

Cropley and Cropley (2005) enriched their hierarchical, four-criterion model of functional creativity with the indicators described above to define a Creative Solution Diagnosis Scale (CSDS). Table 2 shows the full CSDS and Table 3 further illustrates the theoretical framework.

Recognizing creativity in products

The Creative Solution Diagnosis Scale (CSDS) was developed on the basis of a theoretical framework for product creativity, namely the concept of functional creativity (Cropley and Cropley, 2005), enriched by a set of indicators drawn from the literature of product creativity. Three trials of the CSDS have been undertaken and offer some insight into the utility of the proposed scale.

In a pilot-study using the 30-item CSDS Cropley and Cropley (2010b) reported that a small group of 13 teachers (9 women and 4 men) aged from the early 20s to the early 50s used these indicators to assess the creativity of models of wheeled vehicles, designed and built by students, had no difficulty understanding as well as applying the indicators, agreed among themselves (inter-rater agreement), and achieved a reasonable test-retest reliability of 0.79 (Cropley and Cropley, 2010b).

In a more substantial study of the usefulness of the criteria and indicators defined in the CSDS, Haller, Courvoisier and Cropley (2010) used them to rate different designs for a novel, hands-free mobile phone holder made by 55 visual art students at two colleges in Switzerland. These designs were rated by a total of 10 experts (design teachers at the schools in question). The designs were also rated by 5 novices (people with no expertise in design). The median reliability (Cronbach's a) of the overall scores assigned by the 15 judges, for the different designs, was 0.85. Coefficients for the reliability of the total scores for the novice raters ranged from 0.74-0.92; expert ratings (with one outlier removed) ranged, with one exception, from 0.75 to 0.96 (one expert's ratings were noticeably less reliable than those of all others), i.e., the range of reliabilities was almost identical for experts and novices. These findings suggest that long and highly focused experience in the area of design (i.e., extensive exposure to a common design culture) may not be necessary for reliable ratings.

Most recently, Cropley and Kaufman (in press) reported on the application of the CSDS by non-experts, who used the scale to rate a series of five mousetrap designs. In a sample of 203 participants, each rating five designs, the study found a high level of inter-rater reliability (0.96) as well as excellent scale reliability of 0.96.

However, to serve as the basis for a reliable, valid measure of product creativity, and to make a concrete contribution to enhancing the effectiveness of product innovation management, further empirical evidence is needed to demonstrate that the CSDS has high reliability, and yields a valid measure of product creativity. The remaining sections of this paper report on a new study undertaken to test the validity and reliability of the CSDS.

Method

Building on the results of previous studies that used the CSDS to rate a variety of products a new study was undertaken examine the validity and reliability of the CSDS. The 30-item CSDS used by Cropley and Kaufman (in press) was reapplied to a new, larger, group of participants, rating the same five mousetrap designs.

Participants

The participants who used the Creative Solution Diagnosis Scale (CSDS) consisted of 323 college students from a public university in Southern California. Participants took part in the study online for extra credit. The sample included 220 participants who identified as female (68.1%), 52 who identified as male (16.1%), and a further 51 (15.8%) who chose to give no answer for this demographic. The most common age group was 18-24 years old, (79.3%) followed by 25-29 years old (12.6%). The demographic breakdown of the sample was as follows: 124 Hispanic American; 67 European American/Caucasian; 22 African American; 20 Asian American; 11 of mixed or biracial ethnicity and 79 who chose not to identify their ethnicity.

Procedure

Participants were directed to a website where the measures were hosted online. Participants were presented, sequentially, with an image of one of five different mousetraps of varying designs (stimuli 1-5). Images of the mousetraps were selected from Google image search to represent a diverse range of possible mousetraps (see Appendix One for descriptions). Participants were asked to rate each of the different mousetraps using the 30 items of the Creative Solution Diagnosis Scale (CSDS, Table 2). Each item was rated using a 5-point Likert-type scale (ranging from "not at all" through "somewhat" to "very much") to indicate the degree to which the CSDS item applies to the given mousetrap. In addition, each item was rated using a 5-point Likert-type scale (ranging from "not at all" through "somewhat" to "very much") to indicate how creative, overall, each mousetrap was. Participants were asked to complete a basic demographic questionnaire, debriefed, and given extra credit when applicable for their participation.

Prior to commencing data analysis, missing values in the dataset were addressed by excluding data from respondents who failed to complete at least 90% of the total items. This removed 55 out of the 323 respondents (N = 268). Where respondents completed more than 90% of the total items, individual missing data were replaced with the mean of the given indicator. This resulted in a total of 377 replaced missing values in a matrix of 268 x 150 (= 40,200) data items. The total number of replaced missing values was therefore 0.94% of the total dataset.

Reliability

The consistency among the participants was evaluated with Cronbach's coefficient alpha (Cronbach, 1951). Coefficient alpha is a standard measure of internal consistency and has been used in creativity research as a measure of inter-rater reliability, treating raters as items (see Kaufman et al, 2008).

Confirmatory Factor Analysis

Confirmatory Factor Analysis (CFA) involves the specification and estimation of one or more possible models of a factor structure - that is, a structure that relates an underlying construct, in this case Functional Creativity, to a set of variables (individual measures, or items). Any given model (such as that proposed for the CSDS, see table 3) proposes a set of latent variables (factors) to account for covariances among a set of observed variables (Bagozzi, 1980; Bollen, 1989). CFA requires a priori designation of plausible factor patterns from previous theoretical or empirical work; these plausible alternative models are then explicitly tested statistically against sample data. For the present study, four primary factors (Relevance & Effectiveness, Novelty, Elegance and Genesis) provide a framework for explaining the Functional Creativity construct by identifying underlying components that permit more precision in formulating and testing research hypotheses in the field of product creativity. Principle Axis Factoring was selected as the extraction method for its ability to examine shared variance and to uncover the structure of the underlying variables. An Oblimin rotation was used for its assumption that correlations exist between the items of the CSDS (as they are all hypothesized to be aspects of product creativity), in contrast to a Varimax rotation which assumes that items are uncorrelated.

Prior to commencing factor analysis, the data were tested for their suitability for this statistical process using the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (= 0.959) and Bartlett's Test of Sphericity (approximate Chi-Square value = 28417.103, df = 351, significance of p < 0.01). These tests indicated that the data were suitable for factor analysis (Norussis, 1994). The criterion that was applied for accepting an item as loading onto a factor was that the loading should be >0.4. This criterion is consistent with accepted values (see, for example, Comrey and Lee, 1992).

In the present study the factor analysis was commenced with all 30 CSDS items. Items with cross-loadings >0.3 were excluded one at a time, and the factor analysis repeated with the goal of achieving a simple structure in which each item loaded onto a single factor at >0.4.

Linear regression

In addition to rating the artifacts using the CSDS, respondents also gave a rating of overall creativity for each artifact. A stepwise linear regression was conducted for the dataset with Overall Creativity as the dependent variable, and the 27 items of the revised CSDS (after exclusion of three redundant items indicated by the confirmatory factor analysis) as independent variables.

Results

Inter-rater Reliability

The consistency of raters was examined for each individual mousetrap and for the entire dataset, with the following results for coefficient alpha (Table 4):

The consistency of the raters was uniformly high across both the individual stimuli and the entire dataset. The reported values fall into in the excellent range (Nunnally and Bernstein, 1994). As a result of the high reliability of scores across all mousetraps, ratings for the five different stimuli were combined for the purpose of a confirmatory factor analysis of the CSDS.

Scale Reliability

Scale reliability was calculated using Cronbach's alpha to assess the mean inter-item correlations. For the purpose of this analysis a matrix of 1340 (responses) x 30 (items) was used. Cronbach's alpha for the 30 items was 0.96. In both cases this places the overall scale reliability of the CSDS in the excellent range (Nunnally and Bernstein, 1994). The reliabilities of the four sub-scales (the unweighted sums of scores on the items comprising each sub-scale) of the CSDS were also calculated. For the Effectiveness sub-scale (6 items) Cronbach's alpha was 0.91. For the Novelty sub-scale (11 items) Cronbach's alpha was 0.92. For the Elegance sub-scale (7 items) Cronbach's alpha was 0.93. For the Genesis sub-scale (6 items) Cronbach's alpha was 0.92. Once again, in all cases these values fell in the excellent range (Nunnally and Bernstein, 1994).

Confirmatory Factor Analysis

Table 5 shows the simple structure that was achieved after the exclusion of three items (Replication, Incrementation and Combination) that either loaded onto no factor, or cross-loaded at >0.3 and were therefore considered to be factorially impure. In Table 5, for simplicity, only the factor loadings that are >0.4 are shown. The resulting structure satisfies typical criteria for a meaningful and interpretable simple structure, including a minimum of three items per factor. The majority of items in this structure have loadings rated as either "very good" or "excellent" (Comrey and Lee, 1992: >0.7 excellent; >0.63 very good; >0.55 good; >0.45 fair). Finally, the five factors in the simple structure shown in Table 5 account for 66.1% of the total variance.

Linear Regression

Table 6, summarizes the results of the Linear Regression. In the present study nine of the 27 revised CSDS scale items significantly predicted the overall creativity score.

The model summary indicates that 52.2% (R2) of the variance of overall creativity was explained by these nine independent variables.

 

Discussion

The high inter-rater and scale reliabilities of the present study are consistent with those found by Cropley and Kaufman (in press). These figures show that non-expert raters are able to achieve a common understanding of what is meant by the indicators, to recognize them when they see them, and to agree with each other on whether the properties in question are present or absent. Furthermore, these judgments can be expected to be stable over time.

The confirmatory factor analysis that resulted in the simple structure shown in Table 5 strongly supports the theoretical model underpinning the Creative Solution Diagnosis Scale (CSDS) described in Table 3. The six items that load onto Factor 1 are all items identified in the theoretical model as indicators of the criterion "Relevance and Effectiveness". Factor 1 is therefore appropriately named "Relevance and Effectiveness". The items prescription, prognosis and diagnosis loaded onto a single factor. These items are identified in the theoretical

model as "Problematization" indicators, and Factor 2 will be named this way. Five items loaded onto Factor 3 and all of these items can be characterized by the Propulsion Model (Sternberg et al, 2002). Factor 3 will therefore be designated "Propulsion". Factor 4 is characterized by seven items that are associated with "Elegance" in the theoretical model. Finally, Factor 5 is characterized by six items which represent the criterion "Genesis." These results are similar to those found by Cropley and Kaufman (in press), with an identical five-factor solution resulting in both studies.

Table 7 captures the revised version of the Creative Solution Diagnosis Scale (CSDS) derived from the confirmatory factor analysis. After elimination of three redundant or irrelevant indicators from the original 30 (Table 2), the analysis supports the general structure of the theoretical model (Table 3). However, the anticipated factor of "novelty" was differentiated into two components, one involving generation of novelty ("propulsion"), and the other recognition of weaknesses in what exists ("problematization"). This is highly consistent with creativity theory: from almost the beginning of the modern era (e.g. Torrance, 1965) creativity researchers have emphasized finding problems as a key element of the creative process, and some, such as Mumford and his colleagues (e.g. 1996), have examined problem finding in considerable detail. The revised structure therefore offers a reliable and valid instrument for measuring product creativity in terms of a systematic set of criteria derived from creativity theory. The revised CSDS structures of Cropley and Kaufman (in press) and the present study share a common core of 22 indicators across an identical, five-factor structure for product creativity that reflects widely accepted models of creativity.

Conclusions

The process of product innovation is dependent on "product emergence and ... ideation" (Goldenberg and Mazursky, 2002). Management of the process of product innovation requires the ability to measure, control and manage the attributes of a creative product. Aside from a small number of quantitative approaches to creativity measurement reported in this paper, the common approach, used also in product innovation (Goldenberg and Mazursky, 2002) is that of consensual assessment (e.g. Amabile, 1983, 1996). Weaknesses of this approach include its dependence on domain experts, and also the speed and subjectivity with which such judgments are frequently made (Besemer and O'Quin, 1999). To facilitate an effective product innovation process a valid method of measuring product creativity is needed that can be used reliably by non-experts without special training in the assessment procedure.

The revised CSDS is reliable, factorially valid, correlates with global ratings, and is easy to use. It provides managers with a system of indicators for recognizing creativity systematically, and an organized and understandable terminology for discussing or explaining the properties of products. Although it has been used here to rate tangible products, we believe that it can also be applied to concepts, systems, procedures and the like, and provides elements of a "universal" model of the creativity element of the innovative process (see Cropley and Cropley, 2008) . It thus offers product innovation management an important new tool for formulating highly differentiated measures of product creativity that can be used in the development of new products both as a means for stimulating and enhancing creativity and as a diagnostic tool in the process of selecting product ideas.

 

References

ADAMS, R., Bessant, J. and Phelps, R. (2006). Innovation Management Measurement: A Review. International Journal of Management Reviews, 8(1), 21-47.

AMABILE, T. M. (1983). The social psychology of creativity. New York: Springer.

AMABILE, T. M. (1996). Creativity in context: Update to "The Social Psychology of Creativity." Boulder, CO: Westview Press.

BAER, J., Kaufman, J. C. and Gentile, C. A. (2004). Extension of the consensual assessment technique to nonparallel creative products. Creativity Research Journal, 16, 113-117.

BAGOZZI, R.P. (1980). Causal Modeling in Marketing. New York: Wiley & Sons.

BEGHETTO, R. A., Kaufman, J. C. and Baxter, J. (in press). Answering the unexpected questions: Student self-beliefs and teacher ratings of creativity in elementary math and science. Psychology of Aesthetics, Creativity, and the Arts.

BELL, I. A. (1992). Creativity, TV Commercial Popularity, and Advertising Expenditures. International Journal of Advertising, 11(2), 165-183.

BESEMER, S. P. and O'Quin, K. (1987). Creative product analysis: Testing a model by developing a judging instrument. In S. G. Isaksen (Ed.), Frontiers of creativity research: Beyond the basics (pp. 367-389). Buffalo, NY: Bearly.

BESEMER, S. P. and O'Quin, K. (1999). Confirming the three-factor Creative Product Analysis Matrix model in an American sample. Creativity Research Journal, 12, 287-296.

BLEDOW, R., Frese, M., Anderson, N., Erez, M. and Farr, J. (2009). A dialectic perspective on innovation: Conflicting demands, multiple pathways, and ambidexterity. Industrial and Organizational Psychology, 2, 305-337.

BOLLEN, K.A. (1989). Structural Equations with Latent Variables. New York, NY: Wiley & Sons.

CARNEVALE, A. P., Gainer, L. J. and Meltzer, A. (1990). Workplace basics: The essential skills employers want. San Francisco: Jossey-Bass.

CATTELL, J., Glascock, J. and Washburn, M. F. (1918). Experiments on a possible test of aesthetic judgment of pictures. American Journal of Psychology, 29, 333-336.

CHILD, I. L. and Iwao, S. (1968). Personality and esthetic sensitivity: Extension of findings to younger age and to different culture. Journal of Personality and Social Psychology, 8, 308-312.

CHRISTENSEN, C. M. (1999). Innovation and the General Manager, Boston, MA: Irwin McGraw-Hill.

COMREY, A. L. and Lee, H. B. (1992). A first course in factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

CRONBACH, L. J. (1951). Coefficient alpha and the internal structure of tests, Psychometrica 16, 297-334.

CROPLEY, D. H. and Cropley, A. J. (2005). Engineering creativity: A systems concept of functional creativity. In J. C. Kaufman and J. Baer (Eds.), Creativity across domains: Faces of the muse (pp. 169-185). Hillsdale, NJ: Lawrence Erlbaum.

CROPLEY, D. H. and Cropley, A. J. (2008). Elements of a universal aesthetic of creativity. Psychology of Aesthetics, Creativity and the Arts, 2, 155-161.

CROPLEY, D. H. and Cropley, A. J. (2010a). Functional Creativity: Products and the generation of effective novelty. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge Handbook of Creativity (pp. 301-320). New York: Cambridge University Press.

CROPLEY, D. H. and Cropley, A. J. (2010b). Recognizing and fostering creativity in design education. International Journal of Technology and Design Education, 20, 345-358.

CROPLEY, D. H. and Kaufman, J. C. (in press). Measuring functional creativity: empirical validation of the Creative Solution Diagnosis Scale (CSDS). Journal of Creative Behavior.

DAVILA, T., Epstein, M. J. and Shelton, R. (2006). Making Innovation Work - How to Manage It, Measure It and Profit from It. Upper Saddle River, NJ: Pearson Education, Inc.

DAVIS, M. A. (2009). Understanding the relationship between mood and creativity: A meta analysis. Organizational Behavior and Human Decision Processes, 108, 25-38.

DILLON, T. A., Lee, R. K. and Matheson, D. (2005). Value innovation: Passport to wealth creation. Research-Technology Management, March-April: 22-36.

FLORIDA, R. (2002). The rise of the creative class and how it's transforming work, life, community and everyday life. New York: Basic Books.

FREEL, M. (2000). Barriers to product innovation in small manufacturing firms. International Small Business Journal, 18(3), 60-81.

GOLDENBERG, J. and Mazursky, D. (2002). Creativity in product innovation. Cambridge, UK: Cambridge University Press.

GRUDIN, R. (1990). The grace of great things: Creativity and innovation. New York: Ticknor and Fields.

GUPTA, A. K., Smith, K. G. and Shalley, C. E. (2006). The interplay between exploration and exploitation. Academy of Management Journal, 49, 693-706.

HALLER, C. S., Courvoisier, D. S. and Cropley, D. H. (2010). Correlates of creativity among visual arts students. International Journal of Creativity and Problem-Solving, 20, 53-71.

HANER, U.-E. (2005). Spaces for creativity and innovation in two established organizations. Creativity and Innovation Management, 14, 288-298.

HAN, S. H., Yun, M. H., Kim, K. and Kwahk, J. (2000). Evaluation of product usability: Development and validation of usability dimensions and design elements based on empirical models. International Journal of Industrial Ergonomics, 26, 477-488.

HEKKERT, P. and Van Wieringen, P. C. W. (1996). Beauty in the eye of expert and nonexpert beholders: A study in the appraisal of art. American Journal of Psychology, 109, 389-407.

HICKEY, M. (2001). An application of Amabile's Consensual Assessment Technique for rating the creativity of children's musical compositions. Journal of Research in Music Education, 49, 234-244.

HORN, D. and Salvendy, G. (2006). Consumer-based assessment of product creativity: A review and reappraisal. Human Factors and Ergonomics in Manufacturing & Service Industries, 16, 155-175.

HORN, D. and Salvendy, G. (2009). Measuring consumer perception of product creativity: Impact on satisfaction and purchasability. Human Factors and Ergonomics in Manufacturing & Service Industries, 19, 223-240.

JACKSON, P. W. and Messick, S. (1965). The person, the product, and the response: conceptual problems in the assessment of creativity. Journal of Personality, 33, 1122-1131.

KAUFMAN, J. C., Baer, J. and Cole, J. C. (2009). Expertise, Domains, and the Consensual Assessment Technique. Journal of Creative Behavior, 43(4), 223-233.

KAUFMAN, J. C., Baer, J., Cole, J. C. and Sexton, J. D. (2008). A comparison of expert and nonexpert raters using the Consensual Assessment Technique. Creativity Research Journal, 20, 171-178.

KAUFMAN, J. C., Plucker, J. A. and Baer, J. (2008). Essentials of creativity assessment. New York: Wiley.

KIM, W. C. and Mauborgne, R. (2004). Value innovation: The strategic logic of high growth. Harvard Business Review, July-August: 172-180.

LEE, S, Lee J. and Young, C-Y. (2005). A variation of CAT for measuring creativity in business products. Korean Journal of Thinking and Problem Solving, 15, 143-153.

McADAM, R. and Keogh, W. (2004). Transitioning Towards Creativity and Innovation Measurement in SMEs, Creativity and Innovation Management, 13(2), 126-139.

MANN, E. L. (2009). The search for mathematical creativity: Identifying creative potential in middle school students. Creativity Research Journal, 21, 338-348.

MILLER, A. I. (1992). Scientific creativity: A comparative study of Henri Poincaré and Albert Einstein. Creativity Research Journal, 5, 385-418.

MUMFORD, M. D., Baughman, W. A., Threlfall, K. V., Supinski, E. P. and Costanza, D. P. (1996). Process-based measures of creative problem-solving skills: I. Problem construction. Creativity Research Journal, 9, 63-76.

NORUSSIS, M. J. (1994). Professional Statistics, SPSS Inc. Chicago.

NUNNALLY, J. C. and Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.

O'QUIN, K. and Besemer, S. P. (1999). Creative Products, In M. A. Runco and S. R. Pritzker (eds.), Encyclopedia of creativity (pp. 413-422). Boston: Academic Press.

PLUCKER, J. A., Kaufman, J. C., Temple, J. S. and Qian, M. (2009). Do experts and novices evaluate movies the same way? Psychology and Marketing, 26, 470-478.

PUCCIO, G. J. and Cabra, J. F. (2010). Organizational creativity: A systems approach. In J. C. Kaufman & R. J. Sternberg (Eds.), Cambridge Handbook of Creativity (pp. 145-173). New York: Cambridge University Press.

PUCCIO, G. J., Murdock, M. C. and Mance, M. (2007). Creative Leadership: Skills that Drive Change. Thousand Oaks, CA: Sage.

REITER-PALMON, R., Illies Young, M., Kobe, L., Buboltz, C. and Nimps, T. (2009). Creativity and domain specificity: The effect of task type of multiple indices on creative problem solving. Psychology of Aesthetics, Creativity, and the Arts, 3, 73-80.

REIS, S. M. and Renzulli, J. S. (1991). The assessment of creative products in programs for gifted and talented students. Gifted Child Quarterly, 35, 128-134.

ROBERTS, B. (1988). Managing invention and innovation. Research-Technology Management, 33, 1-19.

SAVRANSKY, S. D. (2000). Engineering of creativity. Boca Raton, FL: CRC Press.

SHALLEY, C. E. (2002). How valid and useful is the integrative model for understanding work groups' creativity and innovation? Applied Psychology: An International Review, 51, 406-410.

SIMONTON, D. K. (2009).Varieties of (scientific) creativity: A hierarchical model of disposition, development, and achievement.PerspectivesonPsychologicalScience,4,441-452.

SLATER, B. H. (2006). Aesthetics. Internet Encyclopedia of Philosophy. http://www.iep.utm.edu/a/aestheti.htm#H2 [Accessed February 22, 2011].

STERNBERG, R. J., Kaufman, J. C. and Pretz, J. E. (2002). The creativity conundrum: A propulsion model of kinds of creative contributions. New York: Psychology Press.

STERNBERG, R. J., Kaufman, J. C. and Pretz, J. E. (2003). A propulsion model of creative leadership. Leadership Quarterly, 14, 455-473.

STERNBERG, R. J. (1999). A propulsion model of types of creative contributions, Review of General Psychology, 3, 83-100.

STONE-ROMERO, E. F. and Stone, D. L. (1997). Development of a multidimensional measure of perceived product quality. Journal of Quality Management, 2, 87-112.

TAYLOR, A. (1975). An emerging view of creative actions. In I. A. Taylor, & J. W. Getzels (Eds.), Perspectives in creativity (pp. 297-325). Chicago: Aldine.

TORRANCE, E. P. (1965). The Minnesota studies of creative thinking: Widening horizons in creativity. New York: Wiley.

WEST, M. A. (2002). Sparkling fountains or stagnant ponds: An integrative model of creativity and innovation implementation in work groups. Applied Psychology: An International Review, 51, 355-424.

WHITE, A. and Smith, B. L. (2001). Assessing Advertising Creativity Using the Creative Product Semantic Scale. Journal of Advertising Research, 41(6), 27-34.

WOODMAN, R. W., Sawyer, J. E. and Griffin, R. W. (1993). Toward a theory of organizational creativity, Academy of Management Review, 18, 293-321. Appendix One

Received June 19, 2011
Accepted August 29, 2011

Biographical Notes

Associate Professor David Cropley is deputy director of the Defence and Systems Institute at the University of South Australia. In his role as Associate Professor of Engineering Innovation he has been active in researching creativity and innovation in the engineering domain. Associate Professor Cropley is co-author, with Arthur Cropley, of Fostering Creativity: A Diagnostic Approach for Higher Education and Organisations (Hampton Press, 2009) and co-editor, with Arthur Cropley and James Kaufman, of The Dark Side of Creativity (CUP, 2010).

Associate Professor James Kaufman is director of the Learning Research Institute at California State University, San Bernardino. Associate Professor Kaufman is a widely published author and editor in the field of creativity. His recent books include Creativity 101 (Springer, 2009) and the Cambridge Handbook of Creativity (CUP, 2010).

Emeritus Professor Arthur Cropley has spent more than 45 years researching and writing about creativity at universities in North America, Australia and Germany. He is author of a large catalogue of books and articles on the topic, including Creativity (Longman, 1967), More ways than one. Fostering creativity in the classroom (Ablex, 1992) and Creativity in education and learning (Kogan Page, 2001).

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License