Software startups are challenging endeavours, with various road blocks on their path to success. The current understanding of the challenges that software startups may encounter is very limited. In this paper, we use the research framework of learning and product development stages to analyse the key challenges that software startups have to deal with at different life cycle stages, from problem definition to solution validation and from concept to mature product. Based on an analysis of the empirical data collected by a large survey of 4100 startups, we find out that what perceived as biggest challenges by software startups do vary across different life cycle stages. Building product is the biggest obstacle for software startups, even though its significance decreases when the learning focuses of the startups move from problem to solution and their products mature. Business related challenges such as customer acquisition and scaling are more noticeable at the later stages. Our study raises the awareness of these challenges and suggests to tackle right challenges at the right time.
1 Introduction
Startups are newly created companies that aspire to grow fast in extreme uncertainty. They are considered one of the key drivers of economic growth [1]. But what is also often underlined is the alarmingly high failure rate of startups. Sixty percent of startups do not survive in the first five years, whilst seventy five percent of venture capital funded startups fail [2]. This demonstrates that startups are very challenging endeavours. It is especially true for software startups. According to Sutton [3], software startups are characterized by little or no operating history. Most of them are young and immature. There is a serious lack of time and resource. Moreover, they are subject to multiple influences from an environment that is extremely dynamic, unpredictable and even chaotic. A good understanding of the challenges that software startups have to cope with can help entrepreneurs to be better prepared when confronted by them, and to overcome them eventually.
However, the current Software Engineering (SE) literature offers very limited understanding of the challenges in the context of software startups. A very few number of studies have investigated them in specific areas such as decision making [4], or user experience design [5]. A broader view has been taken in our previous study [6], which examines the key challenges emerging from different areas of early stage software startups. What left unexplored are the challenges faced by software startups at later stages, and how the challenges differ across a startup life cycle. Based on this observation, our study aims at offering a complete and comprehensive understanding of the key challenges in software startups. To this end, we adopted the research framework of learning and product development stages to analyse the key challenges faced by software startups. The main research question asked in our study is:
Anzeige
RQ: what are the key challenges faced by software startups at different learning and product development stages?
To answer the research question, we draw upon the empirical data obtained from a large-scale survey of worldwide software startups conducted between 2013 and 2014. The responses from 4100 software startups were included in the data analysis. The main results of our study is a comprehensive list of challenges faced by software startups at different stages and the contextual understanding of them in terms of learning and product development stages.
The rest of the paper is organized as follows: in Sect. 2, the related work are presented drawing upon relevant software engineering and business literature. Section 3 provides more details on the survey. It is followed by the presentation of the findings in Sect. 4, which are further discussed in Sect. 5, together with the reflection on the limitations of the study. The paper is summarized in Sect. 6 outlining the future research.
2 Literature Review
2.1 Challenges in Software Startups
As Bosch et al. [4] point out, in order to understand the many challenges that software startups face, there is need to understand what a software startup is. An increasingly accepted definition of startup is from Ries [7], a human institution designed to deliver a new product or service under the conditions of extreme uncertainty. This definition highlights the characteristic of no or limited history that a software startup has [3], and the chaotic environment it operates in. However, the definition does not emphasize the intention of a startup to find a scalable and sustainable business model [8], which is a key distinguishing characteristic from established small businesses.
Anzeige
There are few studies that investigate the challenges faced by software startups in SE research field [9, 10], due to the nascent nature of software startup as a research area. One study that touches upon the challenges in early-stage startups is Bosch et al. [4]. One of the two research questions the study explores is what are the typical challenges when finding a product idea worth scaling. They conducted qualitative interviews with the practitioners in nine startup companies. The interviewees confirm that it is very difficult to know how to work in a straight forward manner in early stage startups, and that decision-making support is limited. However no other challenges have been mentioned and the focus of the study itself is less on investigating the challenges and more on developing a methodology to support multiple product ideas being investigated in parallel.
Another study is focused on the specific challenges software startups confront with respect to user experience design, an increasingly important aspect of software engineering. Based on an interview study with eight startups on their approaches to user experience work, Hokkanen and Väänänen-Vainio-Mattila [5] discover several user experience related challenges, including collecting meaningful information from users or customers, applying right method for collecting user feedback, and approaching right set of users.
Our previous study [6] investigated the key challenges faced by software startups at early stages. By “early stage” we mean “from idea conceptualization to first time to market”. Based on a survey study, a list of top 10 challenges were identified: thriving in technology uncertainty, acquiring first paying customers, acquiring initial funding, building entrepreneurial team, delivering customer value, managing multiple tasks, defining minimum viable product, targeting a niche market, staying focused and disciplined, and reaching the break-even point. These challenges are further classified into product, market, finance and team categories. A case study of two software startups is also presented in the paper, to provide a richer understanding than that allowed by a ranking list only. Since the focus of the study is limited to early stage software startups, a complete picture of the challenges faced by software startups at different stages is missing. The study presented in this paper is a continuation of [6] and intends to fill the observed knowledge gap.
2.2 Startup Life Cycle Stages
Learning is a crucial aspect and element for any startup, as emphasized by the Lean Startup methodology [7]. According to Ries [7], startups do not exist to “make stuff”. They exist to “learn how to build a sustainable business”. The process of learning can be divided into four stages in accordance to the customer development process [8]: defining or observing a problem; evaluating the problem; defining a solution; and evaluating the solution. It is worth emphasizing that the learning stages are not linear. Startups need to go through multiple build-measure-learn loops to find their sustainable business models.
On the other hand, a startup goes through a product development process in parallel [8], which can be further divided into the following stages: concept, in development, working prototype, functional product with limited users, functional product with high growth, and mature product. While the learning process is engaged in customer-centric activities mainly happening outside the building, product development is focused on the product-centric activities that are taking place internally. As contended by Blank [8], for a startup to succeed, the two processes must remain synchronized and operate in concert.
In this study, we adopt both learning and product development stages as the perspectives on the life cycle of a software startup, and use them to systematically analyse the perceived challenges.
3 Research Approach
This study is based on a large survey that was employed to explore different aspects of software startups. For the purpose of this study, we only used a subset of the questions in the whole survey. These questions are composed of three parts. In the first part, the respondents were asked to provide background information about their startups, including the principal business domains, the countries they work in, and their roles within their startups. The second part is composed of the questions related to the learning stages and product development stages. Each question should be answered with a single choice from a set of predefined options as described in Sect. 2.2. In the third part, the participants were asked to provide three most significant challenges they perceived recently when working on their startups, ranked as biggest, second biggest and third biggest. Each question in this part should also be answered with a single choice from a set of predefined challenges. To obtain the set of challenges, various online forums related to entrepreneurship were searched. However, one open option was given when each challenge question was asked. If a respondent could not find a suitable option from the list, there was a possibility to specify a different challenge. The list of survey questions relevant to this study can be found in Appendix A.
In total 8240 responses were received. We went through a more strict data cleaning process than that employed in our previous study [6] in order to ensure the quality of the data to be used in the analysis phase. First of all, we filtered out the responses that missed the values in the fields related to learning stage, product development stage and perceived biggest challenge. The data in these fields are mandatory for us to conduct further analysis. Since the unit of analysis is software startup company, we removed the data points which either did not provide startup names or entered suspicious names, such as “balh”, “ABC”, “name”, etc. Secondly, we identified the companies that have multiple responses in the survey, and kept the response from the most senior role of the company based on the assumption that he/she would have a more holistic view of the company. If the roles of the respondents were not provided, we took the last entered entry from the same company. To further clean the data, we removed the responses which did not confirm that the startups in question were still in operation at the time the survey was answered, since the challenges were about those “recently” faced by the startups. If a startup was no more in operation, the answer about the challenges may not be as recent as requested. Last but not least, we removed what we considered “outlier” responses and data points showing some abnormal patterns. As a result, the total sample size was reduced to 4100.
To answer the research question, what are the key challenges faced by software startups at different learning stages and product development stages, we examined the frequency at which each challenge was perceived by the respondents based on the learning stages and product development stages their startups are at. To determine if the challenges perceived by the startups are related to the stages they are at, we formulated the following hypotheses that need to be tested:
H1: There are differences between learning stages regarding the challenges perceived by software startups.
H2: There are differences between product development stages regarding the challenges perceived by software startups.
Since the stages (both learning stages and product development stages) and challenge are categorical variables, to test the relatedness between two categorical variables, Pearson Chi-square test is a suitable statistics. We also checked the expected frequency counts of the cross-tabulation fed into the tests, to make sure that the validity requirements of Chi-square test are met, e.g., no more than 20 % of the cells containing the frequency counts less than 5, and none containing 0 value. We used statistics software package R for both frequency counting and running Chi-square tests.
4 Results
4.1 Background of the Sampled Software Startups
Except the 487 responses that did not reveal locating countries, 3613 sampled software startups come from seventy three countries around the world. Not surprisingly, the majority are located in the United Stages (51.3 %), followed by countries such as Canada (4.98 %), United Kingdom (3.44 %), Israel (2.83 %), Australia (2.61 %), Germany and India (both 2.07 %). The business domains that these startup companies operate in are very diverse and there is no dominant one emerging from the data. The example domains include travel, art and gifts, fashion, e-commerce, social network, idea management, event management, social advertising, project and task management, mobile and social games, luxury hobbies, real estate, e-learning, financial services, health care, etc. The types of software these startups develop are shown in Fig. 1.
Fig. 1.
The types of software applications developed by the startups
×
The typical team size in these software startups is less than 10 people. The most common team sizes are 2 persons (16.3 %), 3 persons (15.6 %), 4 persons (12.2 %), 5 persons (10.6 %) and also, as one respondent put it, “One man army” (9.88 %). In contrast to the very small team sizes, it is interesting to see that the respondents used more than 200 different terms to describe the roles they are playing in their startups. The most frequently mentioned roles are “CEO” (2710), followed by “CTO” (459) and “Engineer” (171). Apart from the traditional chief officer titles, there are also “CXO” (Chief user eXperience Officer), “COO” (Chief Operation Officer), “CPO” (Chief Product Officer), etc. Some interesting titles reflect the characteristics of entrepreneurs, such as “All the hats”, “jack of all trades”, “General Specialist”, “do-it-all”, “all-in-one”, “all rounder”, or “we dont have defined roles”. Others expose nicely the role a founder plays, e.g., “visionary”, “Chief Visionary”, “Chief cook and dish washer”, “motivator”, or “guy that does stuff”.
4.2 Key Challenges Across Life Cycle Stages
The sampled software startups are scattered at the different life cycle stages. As shown in Table 1, in terms of product development stages, the majority are working on either prototypes or functional products with limited users. Only less than 3 % of the startups consider their products mature. In terms of the learning stages, most consider they are in the stage of either validating problems or defining solutions.
Table 1 shows the distribution of sampled software startups across learning stages and product development stages. The biggest percentage are the startups at the stage of defining the solutions and working on functional product with limited users. Two startups are at the problem definition stage but already working on either functional product with high growth or mature product. It might be that the two data points are not valid, or the two startups are truly outliers.
Table 2 lists the challenges perceived by the sampled software startups. It shows that building product is the biggest challenge for 859 startups, the second biggest for 560 and the third biggest for further 327. In total 1746 startups consider it a key challenge. Customer acquisition, funding and building the team are the following big concerns of more than a thousand of startup companies each. In contrast, legal and regulations are perceived as challenges by least startups in the sample.
Figure 2 (generated from the frequency table in Appendix B) depicts how the software startups at one learning stage perceive their biggest challenges differently than those at another learning stage. As shown in Fig. 2, building product as the most frequently perceived biggest challenge is clearly visible, even though the percentage of software startups decreases while the learning stage is advancing.
Another key challenge, the importance of which declines, is minimum viable product. it starts as the third most frequently perceived big challenge at the first learning stage - problem definition. In the solution validation stage, instead, it gives way to other challenges which are much less perceived at the first learning stage, such as critical mass, leadership & team alignment, over capacity/too much to do, and revenue. Staying focused & disciplined shows a similar pattern to minimum viable product.
It is interesting to compare the pair problem solution fit and product market fit. It can be observed that the first fit is a much more perceived challenge than the second at the first learning stage. However its percentage decreases while the startups are focused more on the second fit in later learning stages.
Table 1.
Distribution of software startups across learning and product development stages
Problem definition
Problem validation
Solution definition
Solution validation
Total
Concept
113 (2.76 %)
187 (4.56 %)
118 (2.88 %)
23 (0.56 %)
441
In development
74 (1.80 %)
366 (8.93 %)
331 (8.07 %)
35 (0.85 %)
806
Working prototype
32 (0.78 %)
337 (8.22 %)
358 (8.73 %)
53 (1.29 %)
780
Functional product with limited users
3 (0.07 %)
295 (7.20 %)
1038 (25.32 %)
275 (6.71 %)
1611
Functional product with high growth
1 (0.02 %)
14 (0.34 %)
124 (3.02 %)
202 (4.93 %)
341
Mature product
1 (0.02 %)
11 (0.27 %)
40 (0.98 %)
69 (1.68 %)
121
Total
224
1210
2009
657
4100
\(^{*}\)The percentages are cell percentages.
Table 2.
Overview of key challenges perceived by software startups
No. of software startups that perceive
As 1st challenge
As 2nd challenge
As 3rd challenge
Total
Building product
859
560
327
1746
Customer acquisition
678
454
324
1456
Funding
526
393
420
1339
Building the team
317
394
293
1004
Business model
282
345
250
877
Over capacity/Too much to do
262
309
289
860
Revenue
150
202
326
678
Minimum viable product
130
218
260
608
Staying focused & disciplined
248
191
152
591
Product market fit
151
186
193
530
Critical mass
161
162
132
455
Scaling
92
107
176
375
Problem solution fit
95
100
100
295
Leadership & team alignment
60
99
111
270
Partnership
44
71
114
229
Legal
35
56
61
152
Regulations
10
27
28
65
On the contrary, the percentage of software startups that perceive customer acquisition as the biggest challenge increases along the learning stages. For the software startups at the solution validation stage, customer acquisition exceeds building product noticeably and becomes the biggest challenge for the majority startups at this learning stage. Similarly, partnership and scaling are not perceived as the biggest challenges by the software startups at the first learning stage. They are perceived so by the startups at later learning stages, especially scaling, the significance of which increases greatly at the last stage.
The percentage change of funding takes a different shape. It is perceived as crucial in the first learning stage, but becomes much more noticeable in the problem validation and solution definition phases. Instead, its significance drops back a bit at the solution validation stage. The percentage of building the team challenge does not reveal any obvious pattern of change. Even though fluctuating visibly, it remains as a significant concern across the learning stages. The change of business model does not follow any particular pattern either. However it is visible that the percentage of the startups perceiving it as the biggest challenge drops significantly from the first learning stage to the rest of the learning process. Legal and regulation remain as the least perceived challenges across the learning stages.
Figure 3 (based on a frequency table similar to the one in Appendix B, with product development rather than learning as the stage) depicts how the software startups at one product development stage perceive their biggest challenges differently than those at another product development stage. The challenges in Fig. 3 show less regular patterns when the stages are more granular. But some similar tendencies are still observable, such as building product decrease vs. customer acquisition increase. There are a couple of noticeable differences. One is the percentage change pattern of funding. Even though the significance drops as in the learning stage figure, it is more significant at the early stages of product development, especially at the development and prototyping stages, which is understandable since the companies have no products to sell therefore need funding to sustain the product development. Another notable difference is that legal as the biggest challenge is not perceived by any of the startup companies with mature products.
To test the hypotheses that there are differences between different learning stages (H1) and product development stages (H2) regarding the challenges perceived by the software startups, we run the Chi-square tests on the datasets (see Appendix B as an example). The results are shown in Table 3. With p-value < 0.0001, H1 and H2 are supported with high confidence. We repeated the Chi-square tests on the second and third biggest challenges perceived by the software startups at different stages of learning and product development. They are also significantly related to the learning and product development stages. Therefore H1 and H2 are supported again by taking into account the second and third biggest challenges.
5 Discussion
Table 2 adds more perceived big challenges to the list reported in [6]. The new entries are revenue, scaling, problem solution fit, leadership & team alignment, partnership, legal and regulations. Among them only problem solution fit is clearly a concern more relevant to early stage startups [11]. In comparison to the key challenges in early stage software startups reported in our previous study [6], the list of top ten challenges in Table 2 does not differ much1. The only change is that revenue, in the place of critical mass, becomes one of the top ten key challenges across the stages. The little variance between the two lists can be explained by the fact that the sample used in this study, even though including all stages of software startups, is skewed towards early stage startups. Table 1 shows that the majority of the startups in the sample are at early stages (at the first four stages of product development).
Our study results demonstrate that building product is the biggest challenge faced by software startups at all stages, not just those at an early stage as shown in [6]. Along the same line of argument in [6], this finding is consistent with the generally innovative nature of software startups who are often chasing new technological changes and disrupting the software industry. Therefore they need to deal with cutting edge technology and apply innovative tools and techniques, which renders product development challenging endeavours.
With an extremely small p-value (<0.0001), the hypotheses H1 and H2 are supported, which means that what are perceived as the biggest challenges by software startups do vary across different learning as well as product development stages. Even though it is difficult to declare a global change pattern based on Figs. 2 and 3, it is noticeable that the significance of product and finance related challenges, such as building the product, minimum viable product and funding, decreases when learning and product development progress. In comparison, market related challenges such as customer acquisition and scaling become increasingly perceivable. This is hardly surprising since the main focuses and tasks of startups shift along their life cycles, so do the concerns and challenges entrepreneurial teams have to tackle. The picture is less clear when people and team related challenges are concerned, including building the team and stay focus & disciplined. There is no detectable overall tendency. This is somehow contradictory to our expectation that the more advanced startups are, the more stable and better jelled entrepreneurial teams are, and therefore the less people and team related challenges are perceived.
Fig. 2.
Distribution of software startups in terms of the biggest challenge per learning stage
Fig. 3.
Distribution of software startups in terms of the biggest challenge per product development stage
Table 3.
Chi-square test results
H1(learning stages)
H2(product development stages)
X-squared
506.9612
943.4645
df(degree of freedom)
48
80
p-value <
0.0001
0.0001
×
×
In addition, our data analysis reveals that, in a few software startups, learning and product development stages are not synchronised (e.g., as shown in Table 1), or they are dealing with challenges that are either too early or too late to confront in terms of what need to be learnt or what need to be developed, e.g., confronting product market fit at the problem definition phase (the first learning stage), or still tackling problem solution fit when the product is already mature. As argued in [11], investing on product market fit strategies prematurely given that users are not yet sold on the product can be a crucial failure factor. On the other hand, having already a mature product is a huge waste if the problem solution fit is not reached.
Regarding the limitations of the study, one limitation lies in the set of predefined challenges used in the original questionnaire design. It is not based on existing literature due to the scarcity of related studies. The challenges were obtained through searching various online entrepreneurship forums. They need scientific evidence to support their validity. The fact that most survey respondents selected from the predefined set to certain extent demonstrates that these challenges are relevant and significant. Of course, the fact that no meaningful new challenges were identified in addition to the predefined list may also due to the questionnaire design. A more flexible design would encourage respondents to express the challenges in their own words, even though it means much more effort needed for data analysis. Another limitation of the study is that the life cycle stages of the software startups in the survey were chosen by the respondents, therefore were based on their opinions rather then objective evidences. There could exist inconsistency between the real stage of a software startup and the perceived stage by its respondent. There are additional questions in the original survey (not included in this study) that could be used in the follow-up studies to triangulate the perceived life cycle stages. Lastly, regarding the Chi-square test, it is recommended that the categorical variable has a small number of categories. To improve the confidence of the results, the challenges could be classified into fewer groups. A meaningful and valid way to group these challenges is needed.
6 Conclusions
Software startups are challenging endeavours. Different challenges occupy the central attentions of entrepreneurial teams at different stages. In this paper, we extended the narrow focuses of previous studies and examined the key challenges that software startups have to deal with at different learning stages from problem identification to solution validation, and at different product development stages from concept to mature product, based on a large survey study. We established a ranked list of top challenges, and demonstrated how they vary across different stages.
The findings can guide future studies to address the top software engineering challenges faced by software startups, such as building software product, defining minimum viable product and building entrepreneurial team, while taking into account contextual factors, e.g., the product development stages and learning stages. The practical value of our study is that it raises the awareness of the challenges entrepreneurial teams may encounter and suggest them tackling right ones at the right time.
The survey data provides a snapshot of the challenges faced by different software startups at different stages. A longitudinal study of different challenges faced by same companies at different stages would validate the findings from this study and provide richer contextual understanding of these challenges. It is also interesting to understand the uniqueness of the software startup challenges and their significance in comparison to other types of startups or new product development endeavours in general. Further more, future studies can investigate the potential linkage between the misalignment of learning and product development stages and startup failure.
<SimplePara><Emphasis Type="Bold">Open Access</Emphasis> This chapter is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.</SimplePara>
<SimplePara>The images or other third party material in this chapter are included in the work's Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work's Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.</SimplePara>
The names of the challenges reported in [6] were the adapted versions of the ones reported in this paper. The purpose of the adaptation was to better reflect the characteristics and focus of early stage software startups.
