Machine learning implementation in small and medium-sized enterprises: insights and recommendations from a quantitative study

The term ML, which is often applied in combination with artificial intelligence, is related to the excessive complexity and uncertainty of the new techniques [15]. Out of respect for the complexity, in many cases the technology is not implemented, perhaps because the benefits appear to be less than the potential gains. To generate realistic risk-opportunity awareness, a profound understanding of the strengths and weaknesses of ML technology is necessary.

Two main experiences are distinguished, one is the internal experience, and the other is the external experience. The main part of the internal experience is the existing experience with available data in the context of the respective ML application, which is also described as a success factor of central relevance for the implementation of further machine learning applications [16]. In addition, there is the external experience, which describes the recourse to external knowledge bases. This serves as a common entry point into ML and is, consequently, equally crucial. This includes, for example, the exchange of know-how with non-competing SMEs and companies that have specialized in this subject area [16]. Through in-company experience, SMEs can develop an in-depth knowledge of best practices, challenges, and potential solutions. They thus know competencies and existing relationships that can be leveraged for implementation. Organizations with experience have a deeper understanding of best practices, potential challenges, and effective strategies. This knowledge can inform decision-making, guide the implementation process, and increase the likelihood of success. By knowing about the different existing levels of experience in a company as well as the assumption that the use of ML is probably related to the existing knowledge in a company, the following hypothesis arises:

The experience of the beneficial value, such as more efficient processes, productivity increase or cost reduction, is not the sole prerequisite for ML implementation. It also requires an awareness that implementation cannot be achieved without financial expenditure, sometimes substantial, for which there must be a willingness to pay. The definition of willingness to pay was created to provide a reference for investing in ML. Large companies, which have greater financial resources at their disposal, often pay more for capital in ML technologies and this favors and reinforces the higher usage of ML in comparison to SMEs [17]. There is also a fundamental correlation between the costs of innovation and the probability of introducing it [18]. The less cost-intensive the implementation is, the higher the probability of willingness to pay for it [19]. However, new technologies with potential often require an initial investment that yields returns only after a specific period of productive use. Accordingly, a company must be willing to make the necessary investment in new technology. If the willingness to pay does not exist, no new technology like ML will be implemented. The hypothesis is therefore as follows:

In addition to the existing wealth of experience in a company and the willingness to pay for innovation, the ability to mine data is an important factor in the use of ML. Within this study, Data Mining Readiness of a company was defined as a combination of psychological and structural readiness. Here psychological readiness describes the beliefs and attitudes of organizational members [20]. Psychological readiness is essential for generating and analyzing company data, while structural readiness is more significant in direct comparison [20, 21]. For example, a corresponding open-mindedness and willingness to deal with hurdles and improvements in the context of implementing machine learning are decisive [16]. Generating high-quality data therefore seems to be of crucial importance for the implementation of ML as well as the meaningfulness of the results. Structural readiness, on the other hand, describes the existence of sufficient data volumes, given IT infrastructure, and the availability of tools, which are indispensable for the implementation of ML approaches [21]. Therefore, it can be inferred that data mining readiness has a higher positive impact on the implementation of technologies such as ML and thus the third hypothesis is:

For the study, the machine learning importance was created as a separate factor that might influence the probability of SMEs to implement ML. Here, the focus is on the expected added value, which SMEs can expect from ML. The expectation can be higher at the personal level and at the corporate level, the higher the expected added value, the greater might be the chances of a successful ML implementation. As soon as incentive values in the form of success are missing, no willingness to deal with a topic can be expected [22]. Consequently, to enhance the likelihood of adoption, SMEs need to perceive the introduction of ML as promising and economically viable for their business [21]. Thus, it is assumed that the subjectively perceived importance of ML, therefore, has an influence on implementation of ML in SMEs. This results in the fourth hypothesis derived as follows:

The online survey method was chosen to collect the data relevant to the research interest. The survey was created with the help of the SoSci-Survey tool. Based on extensive literature research, a formulation of content-relevant questions was feasible, which completely maps the dependent variable, the criterion implementation probability of the four independent variables, and the predictors: EXP, WTP, DMR, and IMT (see Table 1).

These formulated questions, hereafter described as items, were combined in a questionnaire (see Table 2). The questionnaire comprised 21 items, with 13 items relating directly to the above variables. The remaining eight items were intended to generate data from participants and companies, such as the age of the participant or the industry of the company. These are marked as PQ in Table 2. Predominantly 5-point Likert scales were used since these make personal attitudes measurable in detail due to their multilevel nature and have therefore also established themselves sustainably in science [23]. The different items were assigned to the predictors to guarantee a manageable analysis. The items EXP1—engaged ML, EXP2—current use of ML, and EXP3—ML competencies measure the construct EXP, the items WTP1—budget, WTP2—external service, and WTP3—internal staff the construct WTP, the items DMR1—employees, DMR2—departments, and DMR3—self equipped the construct DMR and the items IMT1—general importance, IMT2—potential areas and IMT3—relevant use cases measure the construct IMP. The target item IMPL measures the implementation probability of ML. The following Fig. 1 summarizes the initial overall framework, including the one dependent variable, the four predictors and the underlying hypotheses, as well as the various items.

An overview of the criterion as well as the predictors and the associated questions is listed in Table 2 below. According to Allen (2022), the formulated questions must be additionally reviewed by scientists to ensure content validity [24]. Content validity was verified by four scientists as well as four experts from the industry.

The data from the web survey was collected in the period from March 2023 to September 2023. Overall, there was a response rate of 20% of the international SMEs contacted, which amounts to a net sample size of 60 participants.

Table 3 provides an overview of the subjects and company data of the present sample. These data show that there is a broad diversification of the 56 companies regarding their industrial sectors. In terms of sectors, mechanical engineering and plant construction is the largest sector (43%), after that automotive (14%), the electrical industry (11%), and the metal industry (5%) as the smallest sector. Further industries such as for example medical, financial, and mining industry were combined in others (15%). The evaluated parameters also provide information on the turnover and the number of employees, which are within the definition of an SME. The general conditions for an SME, with a maximum of 249 employees and a maximum turnover of 50 million euros, are met. In summary, the responses collected represent a valid sample of SMEs covering a wide spectrum of industries.

The data collected via the online survey was entered into an Excel spreadsheet and then analyzed with the program SPSS 28.0.1.1. In the first step, the dataset was checked for missing data. The item WTP1, which deals with the topic of Engagement on ML, had to be excluded from the analysis, as only 30% of all respondents answered the corresponding questions. All other items were completely answered by all participants, so no further action was necessary due to missing entries. Subsequently, an exploratory factor analysis was conducted to understand which items can be statistically combined into one factor [14]. In general, items with high factor loadings from the exploratory factor analysis were considered more representative of a given factor, indicating a stronger correlation to IMPL. Due to the problem, that exploratory factor analysis results could not be meaningfully interpreted in terms of content, the single-item construct approach was used, to consider the issue of multicollinearity. Multicollinearity can occur in the context of MLR, and the variance inflation factor (VIF) was used as a measure to test for multicollinearity. Because high multicollinearity can distort the results of a regression analysis, making it difficult to isolate the effect of individual predictors, factors with a VIF value greater than 10 were excluded from the regression analysis [25]. Therefore, due to their VIF values exceeding 10, the items EXP2, EXP3, WTP3, DMR1, DMR3, IMT2 and IMT3 were excluded from the regression analysis. This exclusion was necessary because these items had a high degree of overlap in their conceptual content, i.e. they essentially measured similar or redundant aspects of the predictors, potentially biasing the results of the analysis.

After the re-run, the VIF values for the remaining factors were as follows: EXP: 2.24, WTP: 1.65; DMR: 1.74; IMT: 2.13. The highest value of a significant predictor is 2.24, which is marginal and in an acceptable range, below a VIF value of 10 [25].

Only the four items EXP1 (engaged ML), WTP2 (external service), DMR2 (departments in the company), and IMT1 (general importance of ML) were found to be statistically significant and independent of other items after the explorative factor analysis (see Table 4).

Therefore, according to Hair's (2010) best practice, all requirements to apply MLR to a dataset have been fulfilled. Filtering out the confounding variables is essential to represent, describe, and interpret the system as accurately as possible [14]. The confounding variables are typically either the actual pattern of the dataset or significantly affect the regression results, skewing the results. Accordingly, the more precisely a system can be described, the better it can be controlled [26].

First, outliers were identified utilizing an analysis of the standardized residuals. As part of this review, four observations were removed that exceeded the threshold of 1,96 (the critical t value at the 0.05 confidence level) and were thus identified as outliers. The value of 1,96 is widely used in the literature when considering standardized residuals for small to medium sample sizes [14]. For sample sizes (of 50 or more), the standardized residuals approximately follow the t-distribution, such that residuals exceeding a threshold of 1.96 (the critical t-value at a 0.05 confidence level) are considered statistically insignificant. The reduction in the number of valid participants from 60 to 56 after excluding outliers does not represent an exceedance of the recommended sample size for MLR, which must have a minimum sample size of 50. An important parameter for the detection of further outliers is the Mahalanobis distances. To analyze the Mahalanobis distances, the data is examined for leverage values, which are a measure of how far the value of an independent variable is from other values. A general rule for a leverage value is that the highest leverage value is twice as large as the lowest value [14]. In this empirical study, there was no evidence of leverage values, which is why no other outliers can be identified and the final sample size remains at 56 subjects. Finally, Cook distances were used to test the model for negative influential observations. According to Hair (2010), no Cook distance of more than one can exist, or there is a negative influence. However, no observation was found that had a Cook distance of more than one. Consequently, there is no reason to assume further influential confounding variables.

Non-response bias occurs when those who do not participate in a survey differ systematically from those who complete the survey in a way that is significant to the research study. Here, Armstrong and Overton's (1977) recommendation that the sample be divided into early and late responders was followed [27]. These are then tested for differences with known information about the population, such as age, using a T-test [27]. In this case, no significant differences were found, and thus, non-response bias was considered as not present (p = 0,836).

Summarized, the 60 participants were reduced to 56 to be considered within the empirical study by filtering out outliers.

Several important steps were taken to ensure the validity of this study. To begin, content validity was checked, and multicollinearity and bivariate correlations were also excluded. These were examined in the previous chapter as a limitation to conducting the regression analysis. In addition, the residuals are examined below to identify any erroneous data points that may be present, since in this case, endogeneity bias would negatively affect the analysis. The procedure that has proven successful for the residual analysis is that of Pagan and Hall (1983) [28]. Four essential criteria are considered in particular: Linearity of the relationship between criterion and predictors, the constant variance of the residuals (homoscedasticity), normal distribution of the residuals, and independence of the residuals (see Fig. 2). The test is performed with the help of statistical and graphical diagnostics and the residuals were standardized; this makes the residuals more comparable.

The bivariate correlations from the Pearson correlation matrix (Table 5) were used to test for excess collinearity. This test method is limited to the jointly explained variance range of the criterion of only two predictors. In this case, all combinations of predictors were below Hair's critical threshold of 0.7 [14]. Therefore, disproportionate collinearity could be rejected (Table 5). The system has the same accuracy for all reference values, so there are no problems with the linearity of the overall system. In this empirical study, the F-test yielded a high value of 26.2, which allowed the system to be declared statistically significant. In addition, 65% of the variance of the predictors was explained during the examined combination of independent variables (Corr. R² = 0.65). This showed that the result did not arise by chance, nor was there a negative correlation between the criterion and the predictors. The information on the individual parameters is summarized in Table 5.

Because all necessary requirements have been met, the model can be statistically analyzed using MLR.

Within this model, WTP (beta = 0.33; p < 0.01), DMR (beta = 0.27; p < 0.05), and IMT (beta = 0.44; p = 0.001) with a positive correlation to the implementation probability of machine learning in SMEs proved to be statistically significant, thus the null hypothesis can be confirmed. This means that the observed effects of the predictors WTP, DMR, and IMT on the observed criterion IMPL are significant and therefore a practical relationship exists. A beta value of 0.33 for the WTP predictor implies that each one-unit increase in WTP increases the probability of implementation by 0.33 times the standard error of WTP. Similarly, a one-unit increase in DMR increases the probability of implementation by 27% of the corresponding standard error. Similarly, an increase in IMT by one unit leads to an increase in the probability of implementation by 44% of the standard error of IMT. The predicator EXP (beta =  − 0.07 p = 0.55) has a negative beta value, which means that the probability of implementing machine learning in SMEs decreases slightly with each unit increase in EXP. However, this effect was not considered statistically significant, as the threshold value of p > 0.05 was exceeded here. Due to the non-significant values of EXP, no statistically proven statements can be made. The null hypothesis is therefore rejected due to the non-significant influence.

To emphasize the practical implications and relevance of the predictors, this analysis was enriched by extending the focus beyond the statistical significance of the findings. Therefore, a detailed examination of how the predictors align with the initial hypotheses was undertaken, providing a more nuanced understanding of the study's practical implications.

Answer to H2 Hypothesis 2 deals with the WTP and could be confirmed. Likewise, if the companies are willing to take a possible risk of an investment, i.e., the impetus for investment is high, the more likely they are to invest in ML solutions (WTP, beta = 0.33; p < 0.01). The investment of capital on the part of SMEs in necessary technologies as well as in the further training of employees is assigned a high priority and ranks second. This finding may guide SMEs prioritize investments in technology and staff training.

Answer to H3 Hypothesis 3 could be confirmed as well. If the SMEs have an open-mindedness and willingness to deal with hurdles and improvements in the company, the implementation of ML is likely. (DMR, beta = 0.27; p < 0.05). This highlights the importance of a proactive and adaptable organizational mindset and can help SMEs to encourage a working environment that embraces innovation and change.

Answer to H4 Finally, the last hypothesis 4 was also confirmed as true. The IMT (IMT, Beta = 0.44; p = 0.001) has the strongest positive correlation and is, therefore, to be verified as the most important confirmed hypothesis for the implementation. Accordingly, as soon as the introduction of ML is seen as promising for the company and thus the expected value creation increases, the likelihood of using ML technologies also increases. SMEs could use this finding to evaluate and improve their perception of ML, aligning them with their strategic goals to increase the likelihood of a successful ML implementation.

In summary, it can be said that three of the four hypotheses from the empirical study have been confirmed in industrial practice. Willingness to pay, data mining, and the importance of ML are necessary foundations for the implementation of ML technologies. The IMT has the greatest effect on SMEs with a beta of 0.44. Followed by the DMR with high quality and quantity of given data in the company. So DMR is suitable for the use of ML with a beta of 0.33. Finally, and almost as meaningful as the DMR, it can be said that an SME must be willing to make high investments in new technologies. The WTP has a beta of 0.27. Furthermore, these hypotheses are in synergy with each other. The higher the perceived importance, the more the topic will be addressed, and the more data will be collected for a possible implementation. In addition, the higher the expected added value and the higher the chances of success, the more likely it is that investments will be made in ML technologies.

H1, on the other hand, must be regarded as falsified, since no statistically significant correlation between EXP and the implementation probability of ML in SMEs could be established in the context of the study.

In conclusion, hypotheses 2, 3, and 4 were confirmed as significant by the model, only hypothesis 1 had to be falsified.