Implicit theories of intelligence in STEM education: perspectives through the lens of technology education students

As discussed, participation in this study was voluntary and not all students responded to the survey instrument. In this experiment a total of 205 students responded to the survey meaning that results from this sample have a margin of error of ± 4.81% at the 95% confidence interval. A full breakdown of the participants for this experiment is provided in Table 1.

The survey for this experiment was anonymous. The first part consisted of an initial set of questions to gather the demographic information presented in Table 1. The second part of the survey contained one question which asked participants to “list all behaviours characteristic of intelligence in the context of STEM (Science, Technology, Engineering and Mathematics) education”. This question was designed to reflect the intent of Sternberg et al.’s (1981, p. 40) initial question which asked participants to “list behaviors characteristic of intelligence, academic intelligence, everyday intelligence, or unintelligence” however it added the contextual element of STEM education. The survey was created electronically and distributed individually to all students within the cohort.

As participants were listing behaviours, minor variations in language emerged. For example, the characteristic of “problem-solving” was frequently cited with variations such as “the ability to solve problems”, “problem solving ability”, and “problem-solving skills”. Therefore, prior to further analysis, all listed characteristics were coded to remove duplicates emerging from minor variations in language.

In this experiment a total of 213 students responded to the survey meaning that results from this sample have a margin of error of ± 4.62% at the 95% confidence interval. While this experiment contains a different sample than Experiment 1, the participant population remained the same. A full breakdown of the participants for this experiment is provided in Table 4.

The survey for this experiment was anonymous. The first part consisted of an initial set of questions to gather the demographic information presented in Table 4. The second part of the survey contained the list of 84 behavioural characteristics generated from Experiment 1 with one question which asked participants to “rate how important each of these characteristics are in defining ‘your’ conception/understanding of an intelligent person within STEM education”. The order of the items were randomised for each individual participant to prevent the occurence of an order bias. Each behaviour was rated on a 5-point Likert scale with the ratings “1—Not important at all”, “2—Unimportant”, “3—Neither important nor unimportant”, “4—Important”, and “5—Very important” (Cohen et al. 2007). This question was designed to reflect the intent of Sternberg et al.’s (1981) question regarding how characteristic behaviours were of intelligence, however it again added the context element of STEM education. The survey was created electronically and distributed individually to all students within the cohort.

As the multivariate analyses of exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and structural equation modelling (SEM) utilised to analyse the data from this experiment assume normal distributions and are sensitive to extreme outliers, the data was screened for both univariate and multivariate outliers prior to the conduction of these tests (Kline 2016). Univariate outliers were identified as results which exceeded three standard deviations from the mean. 123 data points (.68% of the dataset) were identified as univariate outliers under this criterion and were transformed to the value equal to three standard deviations from the mean (Kline 2016). Data was then screened for multivariate outliers using both the Mahalanobis D and Cook’s D statistics. The criterion for identifying outliers with the Mahalanobis D statistic was p < .001 (Kline 2016) and for the Cook’s D statistic it was any instance greater than 1 (Cook 1977). While no cases were identified as multivariate outliers under the Cook’s D criterion, seven cases (3.29% of the dataset) were identified as outliers under the Mahalanobis D statistic. These seven cases were excluded from the analysis leaving a total dataset of 206 responses. Additionally, skewness and kurtosis values for all behaviours were within acceptable limits of between ± 2 (Gravetter and Wallnau 2014; Trochim and Donnelly 2006).

Prior to the conduction of any statistical analyses, it was of interest to examine the rank order of behaviours pertinent to how important they were in defining the participants’ conceptions of intelligence. An observation of the standard deviation values suggests a relatively high degree of consensus from the participants. The behaviours listed range considerably in terms of how important they were with a minimum value of 1.831 (Being awkward) and a maximum of 4.516 (Being interested in the subject area). An examination of the lower ranked items illustrates social actions which could be described as negative, inhibitory or nonsocial such as “being awkward” and “being antisocial” are not considered as being important characteristics in defining intelligence with mean scores rising considerably where behaviours transition to neutral and positive traits (Table 5).

Subsequent to examining the descriptive statistics and rank order of the behavioural traits, correlations between year groups, item reliability and subject reliability coefficients were determined (Table 6). All observed correlations were very strong (r = .919– r = .980) (Evans 1996) and were all significant at the p < .001 level. In addition, all reliability statistics were very high with the minimum item reliability statistic being observed within the 4th year cohort (α = .927) and the minimum subject reliability statistic being observed within the 3rd year cohort (α = .974). These results indicate there is a very strong conception of what it means to be intelligent within STEM education within this cohort. The strength of this conception is further emphasised when considering the results of Sternberg et al.’s (1981) study, where correlations among experts ranged from r = .67 to .90 and among laypeople ranged from r = .36 to .81. Interestingly, there appears to be little variance between year groups however considering the strongest correlations are observed between the 1st and 2nd year cohorts (r = .972) and between the 3rd and 4th year cohorts (r = .980) there may be a shift in thinking occurring as the students transition from the initial 2 years to the latter 2 years of study. Finally, considering the strength of these results, it is important to appreciate the reliability statistics from Experiment 1 which highlight the large variance in the participants’ individual understandings of STEM intelligence.

A factor analytic approach was adopted for the final element of the analysis. This included a combination of exploratory factor analyses (EFA), confirmatory factor analyses (CFA) and structural equation modelling (SEM). EFA was selected as the intent of this analysis was to determine underlying relationships between the variables in the dataset (Byrne 2005). Specifically, the maximum likelihood method of extraction was selected as in the data screening stage assumptions of normality were not violated (Fabrigar et al. 1999). An oblique promax rotation was selected as it was hypothesised that the factors would correlate (Osborne 2015).

To determine the factorability of the dataset for the EFA a number of approaches were used. The correlation matrix was examined and revealed that 455 out of 3486 correlations were above .3. The anti-image correlation matrix was examined which showed anti-images correlation for all 84 variables as greater than .5. The Kaiser–Meyer–Olkin measure of sampling adequacy was .821, above the recommended value of .6 (Kaiser 1974), and Bartlett’s test of sphericity was significant (χ² (3486) = 8067.943, p < .000). These criteria suggest a reasonable level of factorability within the data (Tabachnick and Fidell 2007).

To determine the quantity of factors to extract a number of criteria were examined including eigenvalues > 1 (Kaiser 1960), a scree test (Cattell 1966) and a parallel analysis (Horn 1965). Horn’s parallel analysis has been identified as one of the most accurate priori empirical criteria with scree sometimes a useful addition (Velicer et al. 2000). Figure 1 illustrates the scree plot with parallel analysis. An examination of the number of factors with eigenvalues > 1 suggests a 23 factor solution. The result of the parallel analysis suggests a five factor solution. Finally, an examination of the scree plot further corroborates a five factor solution however it suggests merit in examining three and four factor solutions as well. Therefore, EFA’s were conducted with three, four and five factor solutions.

The results of the three EFA’s are presented in Table 7 (five factor solution), Table 8 (four factor solution) and Table 9 (three factor solution). In each instance only variables with a salient loading of > .4 on at least one factor are represented. No variable had a salient loading of > .4 on more than one factor and therefore a simple structure was attained (Thurstone 1947) in each circumstance.

The first two factors of the five factor model (Table 7) appear to represent factors describing ‘social competence’ and ‘general competence’. The social competence factor was named to reflect the factor found by Sternberg et al. (1981). The general competence factor was named to reflect Spearman’s (1904) idea of a general intelligence (g) while preserving the idea that the variables describe a level of competency. The third factor, through its inclusion of craft skill, imagination, and designerly abilities, suggests a discipline specific factor relative to the cohort and was therefore named ‘technological competence’. The fourth factor includes a series of behaviours arguably not conducive to positive social interactions such as being antisocial, regularly procrastinating and being awkward. While being quiet and reserved does not necessarily mean a person is not socially adept, in this instance the behaviour is posited to be reflective of a person who does not actively seek to engage in social interactions. Therefore this factor was termed ‘nonsocial behaviour’ to encompass both a lack of social skills and/or a reservation towards social interaction. The fifth factor, containing only three variables, is difficult to ascribe a name to. While being mathematical and being scientific appear to suggest a distinct type of personality, the high level of literacy factor is arguably representative of many different personas. Therefore, the tentative title of ‘scientific presence’ was ascribed to this factor under the position that in this circumstance, the level of literacy variable was considered to represent a level of technical literacy (Dakers 2006; Ingerman and Collier-Reed 2011).

In the four factor model (Table 8) the factor structure remained similar to the five factor solution however the fifth factor from the previous model (scientific presence) no longer emerged. Considering it only described three variables with two of these no longer having a salient loading of > .4 on any of the remaining factors in the four factor solution, this suggests a five factor solution may be representative of overextraction (Wood et al. 1996). In the three factor solution the previously named technological competence factor no longer emerged. In the five factor model it correlated moderately with the general competence factor (.414) with a similar correlation being observed between the two factors in the four factor model (.487). Many of the variables from the technological competence factor can be observed within the variables of the previously described general competence factor and therefore in the three factor model it was renamed to ‘general and technological competence’ to reflect this change. However, considering the evidence for a cohort specific factor from the work of Sternberg et al. (1981) and the clear distinction between the variables in the general competence and technological competence factors, it is posited that the three factor model is representative of underextracting (Wood et al. 1996).

Developing on the results from the EFA, further analysis was conducted through both CFA and SEM. While the four factor model is the most theoretically sound, both the three and five factor models were also initially examined through SEM to confirm which model best fit that data. In addition to examining these models, the existence of the nonsocial behaviour factor is questionable in terms of how characteristic it is of intelligence. An examination of the mean scores achieved by the variables within it suggests that it is not an important factor (Table 5). Therefore, SEM was conducted on the five and four factor models with this factor excluded. The three factor solution was not examined without the nonsocial behaviour factor as the model would require an additional constraint to make it identifiable. The results of this analysis are presented in Table 10.

A number of fit indices were included to support the interpretation of the model of best fit. These include the relative Chi square statistic (χ²/df) (Wheaton, Muthén, Alwin, and Summers 1977) which should have values < 2 (Tabachnick and Fidell 2007; Ullman 2001), the goodness-of-fit index (GFI) (Jöreskog and Sörbom 1986) which should be ≥ .95 (Shevlin and Miles 1998), the adjusted goodness-of-fit index (AGFI) (Jöreskog and Sörbom 1986) which should be ≥ .90 (Hooper et al. 2008), the comparative fit Index (CFI) (Bentler 1990) which has a cut-off point of ≥ .95 (Hu and Bentler 1999), the root mean square error of approximation (RMSEA) (Steiger and Lind 1980, cited in Steiger 1990) with a cut-off point of ≤ .06 (Hu and Bentler 1999; Lei and Wu 2007), and the Tucker-Lewis index (TLI) (Tucker and Lewis 1973) which is advised to be ≥ .95 (Lei and Wu 2007).

However, models not meeting these cut-off points should not necessarily be rejected as there are degrees of model fit. For the RMSEA, it is suggested that values lower than .08 are indicative of reasonable fit with values lower than .05 indicating good fit, while for CFI values greater than .90 are suggested to indicate reasonable fit and values greater than .95 to indicate good fit (Kline 2005). Based on this, many researchers regularly opt for the cut of .90 as the cut-off point for the GFI, AGFI, TLI and CFI indices and .08 for the RMSEA (e.g. Engle et al. 1999; Kozhevnikov and Hegarty 2001; Maeda and Yoon 2015; Vander Heyden et al. 2016). In addition to this, the GFI and AGFI indices are susceptible effects caused by sample size (Hooper et al. 2008; Sharma et al. 2005) and therefore should not be used exclusively.

An examination of the fit indices for each model reveals that all models meet the χ²/df and RMSEA criteria but no model meets the criteria for GFI, AGFI, CFI or TLI. As model D (the EFA four factor solution excluding the nonsocial behaviour factor) is the best fitting for these criteria, modifications were made by removing observed variables with low loadings on latent factors (Table 11). The approach taken was to remove a small number of variables at a time which loaded below .5 on their respective latent factor. A final model (Model D₄) was examined in which all observed variables had loadings of ≥ .5.

Ultimately no model achieved the criteria for model fit under the GFI and AGFI indices however due to the previously described argument regarding sample size effects this was not regarded as critical to the analysis. After the second round of adjustments (Model D_2(SEM)), reasonable model fit was achieved under the TLI and CFI indices. These were improved upon in subsequent refinements (Models D_3(SEM) and D_4(SEM) respectively). It was decided not to remove any more observed variable to improve model D_4(SEM) as its current structure provided a clear perspective of each latent variable and further reductions would render factor interpretation theoretically difficult. The final models are presents in Figs. 2 and 3 which show the factor loadings on the participants combined implicit theory of intelligence and the covariances between these factors.

Sternberg (1984) postulates the potential for a ‘common core’ of intellectual functions which are culturally shared. This construct is premised on the theory that certain intellectual behaviours are associated more with being human in general than with operating in any specific discipline. Defining the components of this common core as ‘metacomponents’ of intelligence, Sternberg posits them to include recognising the existence and nature of a problem, deciding upon the processes needed to solve the problem, deciding upon a strategy into which to combine those processes, deciding upon a mental representation upon which the processes and strategy will act, allocating processing resources in an efficacious way, monitoring one’s place in problem solving, being sensitive to the existence and nature of feedback, knowing what to do in response to this feedback, and actually acting upon this feedback (Sternberg 1980, 1982, 1984). Considering this theory in conjunction with the results of Sternberg et al.’s (1981) study and the results of this study suggests that, at least from an implicit perspective, the acknowledement of such a common core does exist. Explicit evidence for a common core is offered through the wealth of psychometric research conducted with the aim of eliciting an empirically based objective theory of human intelligence. Of particular relevance is the theory of fluid and crystallised intelligence (Gf–Gc theory). The Gf–Gc theory was first theorised by Cattell (1941, 1943) as an advancement of Spearman’s (1904) idea of a singular general intelligence, g, into the dichotomy of a fluid and a crystallised intelligence. Cattell (1943) conceived his theory of fluid and crystallised intelligences from observations of intelligence tests designed for children and their lack of applicability to adult populations. Synthesising the observations of the adult dissociation of cognitive speed from power and the diminished g saturation in adult intellectual performances with neurological evidence identifying a localised brain legions as effecting children generally while a corresponding legion effecting adults more in terms of speeded tasks, abstract reasoning problems, and unfamiliar performances than in vocabulary, information and comprehension (e.g. Hebb 1941, 1942), Cattell (1943) postulated the potential for general intelligence to comprise of two separate entities. Fluid intelligence is defined as “a facility in reasoning, particularly where adaptation to new situations is required” while crystallised intelligence is defined as “accessible stores of knowledge and the ability to acquire further knowledge via familiar learning strategies” (Wasserman and Tulsky 2005, p. 18). Sternberg et al. (1981) identified factors in both the expert and laypeople cohorts resemblent of fluid and crystallised intelligence factors and the general competence factor apparent in this study also aligns with a fluid intelligence factor. The general competence factor identified within this study is reflective of Sternberg’s idea of a common core with it’s included variables corrseponding to his descriptions of metacomponents. Interestingly, a factor similar to a crystallised intelligence factor was not identified within this study however this is posited to be reflective of the nature of knowledge inherent within technology education.

In order to appreciate the nature of the three factors identified from this study it is paramount that the context of technology education is examined. The results of this study do not suggest a crystallised intelligence factor as being perceived as inherenlty characteristic of intelligence within technology education. Considering crystallised intelligence as associated with having and developing a defined knowledge base, it is interesting that such a factor should not emerge within a specific discipline. While models of technological capability suggest the importance of an appropriate knowledge base (e.g. Gibson 2008), it is arguable that an explicit and defined knowledge base does not exist within the discipline. A reason for this is offered by McCormick (1997) who notes that technological activity is multidimensional, drawing on subjects such as science, mathematics and engineering, and that it is found in all spheres of life making defining an explicit knowledge base very difficult. Instead, McCormick (1997) suggests that explicit technological knowledge will be relative to specific tasks and circumstances. Kimbell (2011) argues that technological knowledge is inherently different to scientific knowledge whereby scientific knowledge is concerned with literal truths and technological knowledge is more aptly associated with usefulness. As design is a core element of technology education, absolute knowledge is not always necessary. Instead, Kimbell (2011) suggests that ‘provisional knowledge’ is more aligned with the discipline and that learners reside in an “indeterminate zone of activity where hunch, half-knowledge and intuition are essential ingredients” (p. 7) in using their provisional knowledge to support further inquiry in response to particular tasks. Aligning with this and recognising that design is embedded within a personal and social context, Williams (2009, pp. 248–249) argues that “the domain of knowledge as a separate entity is irrelevant; the relevance of knowledge is determined by its application to the technological issue at hand. So the skill does not lie in the recall and application of knowledge, but in the decisions about, and sourcing of, what knowledge is relevant”.

The results of this study indicate that a defined knowledge base is not perceived as being inherently important within the discipline. Having empirical data corroborating this perspective from a cohort who are engaging with the domain both as learners and educators is particularly advantageous due to their unique position. From a pedagogical perspective, this presents educators with a particularly interesting challenge in that they need to negotiate the type of explicit knowledge they should teach as a medium for developing the broader intellectual and behavioural traits pertinent within the discipline. Furthermore, the findings from this study serve to facilitate the development of an empirically supported explicit theory as this evidence suggests that domain-free general capacities (Schneider and McGrew 2012) may serve as an auspicious core set of cognitive factors pertinent to technology education. It is important to note that this argument does not aspire to dilute the importance of developing technical expertise within the discipline as an advanced level of expertise in particular areas does typically constitute a developed and refined content knowledge. Instead, it serves to reiterate the position that, at least within technology education, intelligence and expertise can be seen as separable constructs. Intelligence within the discipline is multifaceted, perceived to consist of general, technological and social competencies. Expertise in specific areas is critical however the particular type of expertise developed will be in response to a specific need. Therefore, when considering pedagogy in technology education, as the nature of explicit technological knowledge can be variable. The results of this study are significant in refining the type of general competencies which the teachers should aim to develop while they themselves use their own professional judgement in the selection design and selection of specific knowledge and tasks.

The results of this study do suggest a social competence factor and a general competence factor similar to fuild intelligence. Building on the idea that a defined knowledge base may not exist, these factors further describe the applied nature of technology education. Considering the magnitude of influence that design has within technology education, students regularly encounter novel situations. Fluid intelligence describes the ability to engage in such situations and a recognition of this substantiates the idea that intellectual behaviour in technology education is more concerned with a capacity to react to negotiate within ones environment rather than having acquired highly developed schema. Further considering the nature of technology education, design is recognised as a social activity (Hamilton 2003, 2004; Murphy and Hennessy 2001). Conversation is placed at the core of the educational process (Trebell 2007) where teacher-student and student–student interactions are critical for learning. Specifically within technology education, Murphy and Hennessy (2001) have shown that students seek opportunities to interact with peers even when not explicitly advocated for within the pedagogical approach adopted by the teacher. The social competence factor which emerged within this study reflects this idea and the importance of such social interactions within technology education are reflected in the contemporary agenda to support discourse through the provision of a shared language (O’Connor 2016; O’Connor et al. 2016a, b). Interestingly, a tentatively named nonsocial behaviour factor did emerge. While ultimately its removal did increase the model fit of the data, its initial presence is further suggestive of the importance of social skills within the discipline. It emerged in the results of Experiment 1 as some participants conceived intelligence as synonymous with an antisocial or nonsocial stereotype however the importance of such behaviours were ultimaltey considered unimportant in the cohorts shared conception.

Perhaps of most importance to the discipline is the technological competence factor which emerged in the final model. Its importance to the cohort is shown in the SEM model (Fig. 2) as it has the highest loading (.93) on their implict theory of intelligence. The variables which describe this factor are particularly interesting. While the “having good coordination” variable is ambiguous, it is posited that this was conceived as having a similar meaning to achieving a high level of craft skill. This particular theme is extremely pertinent to the discipline as the philosophical underpinnings of the discipline emphasis the criticality of craft skill and the ‘make’ aspect of the subject.

As technology education is only one of the four disciplines which describe STEM education, it would be of significant interest to conduct similar studies in the areas of science, engineering and mathematics education where the results of each could be synthesised to provide a holistic model of implicit theories within STEM. In particular, it would be interesting to identify if cohort specific factors similar to the technological competence factor would emerge as conceptions of intelligence have a significant role within multiple areas of education. It is posited that a factor similar to the general competence factor in this study would exist within the other disciplines with a factor similar to crystallised intelligence also being probable due to more explicit knowledge bases. One of the most significant implications of this work and for its progression is in its usefulness in developing an explicit theory of intelligence. There have been calls to make education and pedagogy more scientific (OECD 2002) and while many relevant explicit theories exist and are being developed, knowing what is perceived to constitute intellectual behaviour within specific disciplines can add an additional perspective to guide this research to further support teaching and learning practices.

There are a number of limitations which should be considered for interpreting the generalisability of the results and in framing pertinent future research agendas. Firstly, little work has been conducted with this particular methodology to examine prototypical conceptions on intelligence in education since the work of Sternberg (Sternberg 1985; Sternberg et al. 1981). Therefore, while theoretically a person’s belief about the structure of intelligence should influence their educational experience and engagement, further work needs to be conducted to objectively examine such effects. Substantial work in the area of entity and incremental beliefs as previously discussed shows that beliefs about the nature of intelligence have a significant effect however the generalisability of implicit theories cannot be assumed to include the prototypical perspective.

Secondly, this study was conducted with a cohort representative of only one cultural context. In this case the participant population comprised of a cohort of ITE students in one institution who have therefore been exposed to an educational philosophy synonymous with that context. In order to reduce the potential bias associated with an inherent pedagogy, ITE students from other technology education courses in other institutions and countries should be included to support generalisation. In addition, the cohort represents only one group of stakeholders within technology education. As quasi-experts, the knowledge base is not sufficient to provide certainty in an absolute model of theoretical intelligence within STEM education from the perspective of technology education. The perspectives of other stakeholders such as primary and post-primary pupils, primary and post-primary teachers, international technology education researchers and experts, and pertinent governmental representatives should be considered.

Thirdly, the study only considered one discipline from within STEM education and therefore further evidence is required to support the hypothesis of a cohort specific factor. It would be of significant interest to determine if ITE students from science, engineering and mathematics disciplines also conceived a cohort specific factor pertinent to their implicit theories of intelligence and to examine the similarities and variances between these. Combining the implicit theories of intelligence from all disciplines may result in a model containing factors which are appropriate to all disciplines which could be classified generally as perceived core intellectual traits within STEM education. This would support the creation of an explicit theoretical model of cognitive factors pertinent to STEM education through the adoption of empirical and objective measures, with core general factors pertinent to all disciplines and peripheral factors associated with individual disciplines.

Fourthly, as the list of behaviours generated in the initial experiment governed the remit of the second experiment, it may be appropriate for future studies to include an additional data collection phase in the interim. For example, the individual questionnaire from Experiment 1 could be succeeded by a series of focus groups in a quasi-Delphi approach where individual perspectives may act as a stimulus to prompt further thoughts from their peers. This approach would preserve the unbiased nature of the data from Experiment 1 but may increase the number of variables for Experiment 2 and would increase their validity.

Finally, the sample size for the multivariate analyses conducted in Experiment 2 could be considered small. While the sample (n > 200) is sufficient (Tabachnick and Fidell 2007), it would be advantageous to increase this. While the model fit indices from the CFA and SEM analyses suggest this was not a necessity, this would further reduce the biases previously discussed and would add to the statistical strength of the analysis.

The findings of this study provide insight into the way students of technology education perceive intelligence. While the importance of educators understanding student’s implicit theories about intelligence concerning entity and incremental beliefs is acknowledged (Flanigan et al. 2017), it is also of paramount importance for educators to understand what students perceive to be intelligent behaviour. A misalignment between student and teacher expectations has the potential to elicit many negative educational implications and therefore the knowledge of student beliefs can support the development of mutual understandings.

The findings of this study suggest that students of technology education perceive intelligence within the discipline to be multifaceted, comprising of three factors including social, general and technological competences. As these are context specific, it would now be advantageous to synthesise these results with similar findings from other educational disciplines. Within technology education, these results afford educators, researchers and other stakeholders a lens for which to view educational planning and provision. Within the broader remit of STEM, the results of this study have the potential to frame the unique position of technology education. It allows this at a minimum by looking at the technological competence factor. It also facilitates comparisons with additional disciplines by looking at the other factors as well.