Forecasting unemployment with Google Trends: age, gender and digital divide

The world has undergone a dramatic change with the rise of the Internet in the twenty-first century. Particularly, job seeking has been strongly influenced, being carried out increasingly by online resources. In fact, this job searching method has become the most common one, used by employed and unemployed people, as it is expected to improve the chances of finding a job while considerably reducing searching costs. For instance, in 2009 more than 70% of US young unemployed searched for a job online (Khun and Mansour 2014). In Spain, a recent survey among unemployed people revealed that 98% of them use Internet searches and job search websites (Adecco 2016).

Simultaneously, in parallel with the increasing use of the Internet, the digital revolution makes available for researchers a huge amount of data that can be exploited to produce more accurate predictions of countless variables, including unemployment figures. In the last decade, numerous studies employ data mined from the Internet searches to improve the predictions obtained with most common models. Pioneer papers applying this approach arise in the field of medicine see, e.g., Johnson et al. (2004), who analyze the relationship between Internet searches for flu symptoms and the number of cases reported in the USA. In the last decade, this idea has produced a fruitful literature with, in particular, many papers focused on predicting the evolution of the labor market. Some examples are the studies by Choi and Varian (2009, 2012); Pavlicek and Kristoufek (2015); Niesert et al. (2020) and Caperna et al. (2020). Other papers, closer to ours, will be discussed in the next section.

The queries in the search engines, in our case Google Trends (GT), are free and easily obtained. They also offer broader and more up-to-date data than commonly used surveys, which are released with some delay. Yet the data mined from Internet searches are far from being the panacea. In this regard, Cebrián and Domenech (2022) evidence non-negligible issues related to its measurement accuracy. Similarly, Naccarato et al. (2018) point out that the unemployment data cover a known population with an estimated and reliable error distribution. In contrast, data downloaded from GT are not a probabilistic sample of the population and so, its error distribution is unknown. Researchers should keep in mind that its representativeness is closely linked to the consumption patterns and Internet penetration rate. In this sense, only with perfectly global spread of the Internet and same usage patterns by age and gender, data mined from search engines would be thoroughly representative.

Concerning this last point, numerous studies have addressed the issue of the so-called digital divide, regarding the differences in the uptake of Internet access and usage patterns by age, race, gender and socioeconomic status (see, e.g., Novak and Hoffman 1998; Enoch and Soker 2006; Abbey and Hyde 2009; Hidalgo et al. 2020). The reader is referred to van Dijk (2020) for a complete and up-to-date survey on this topic. Particularly, Gómez (2019) presents the asymmetries in terms of access and usage to the network in Spain. As expected, usage is higher the higher is the level of education and the better is the economic situation, although these gaps have been considerably reduced in the last decade (see, also, Cañón Rodríguez et al. 2016). To monitor this, the Spanish National Institute of Statistics releases a report on the digital gap by groups of age, education levels and gender (see INE 2020).

The paper is organized as follows. Section 2 provides a revision of the literature in the use of GT as predictors, focusing on unemployment applications and digital divide. Section 3 details the data employed in the analysis, paying particular attention to the GT queries and how those are generated. The benchmark model and the proposed alternatives are presented in Sect. 4. Section 5 compares the forecasting results of the proposed models relative to the benchmarks for all the combinations of gender and age group. A discussion and some concluding remarks close the paper.

The available (quasi) real-time GT data allow nowcasting models, which provide more accurate estimates than those generated by conventional indicators. This has been shown by many authors for unemployment figures, and other variables related to the labor market, in countries such as the USA, Germany, Romania, the UK, Spain, France, Italy or Canada.

Focusing on the USA, Choi and Varian (2009, 2012) and Nagao et al. (2019) demonstrate that introducing an indicator of the number of searches in Internet improves the results of conventional models when predicting the unemployment. D’Amuri and Marcucci (2009) provide similar results, in this case by applying an index generated by searches in GT. Later, the same authors revisit the theory of their previous work, disaggregate the GT searches at a federal level and incorporate the effects of the 2008 Great Recession in D’Amuri and Marcucci (2017). More recently, Borup and Schütte (2022) analyze the impact in the forecast when using a large amount of GT-query variables. They conclude that GT variables do not seem to be better predictors for the unemployment than the classical macroeconomic and financial series. However, combining many GT series, preferably with nonlinear procedures, increases the forecasting power, significantly overtaking the above-mentioned classical indicators.

For Germany, Askitas and Zimmermann (2009) compute the improvement of the unemployment forecasts using three groups of keywords (unemployment agency, unemployment rate, staff consultant) and a set of queries linked by the Boolean operator ‘OR,’ captured from the job vacancies websites Monster and Jobboerse.

The effect of GT searches at a regional level has been studied by Simionescu (2020). This paper’s main contribution is to analyze the results of this methodology when forecasting the unemployment rate in Romanian counties, which are heterogeneous in terms of economic and social development. In a subsequent paper, Simionescu et al. (2020) estimate the impact of the Brexit on the unemployment for the UK also with GT predictors.

Regarding Spain, several studies have arisen recently. Vicente et al. (2015) and González-Fernández and González-Velasco (2018) obtain better predictive accuracy when forecasting the Spanish unemployment with two and one GT searching terms, respectively, than with univariate and multivariate models that do not include this information. Similarly, Mulero and Garcia-Hiernaux (2021) find a remarkable precision gain when forecasting monthly unemployment with numerous GT-query searches and dimensionality reduction techniques. Finally, Simionescu and Cifuentes-Faura (2022) also demonstrate the capacity of GT data to predict Spanish and Portuguese unemployment (at a regional level), in this case by means of dynamic panel data models.

As mentioned, D’Amuri (2009) applies the same idea to beat the official unemployment forecast in the USA. Interestingly, this author is the first to focus on a potential selection bias, finding a digital divide effect in favor of younger people, arguing that they are the greatest consumers of Internet. In contrast with the considerable amount of papers dealing with the unemployment prediction gain when using GT predictors, only a few focus on this selection bias related to digital divide. Some of them are Fondeur and Karamé (2013) and Naccarato et al. (2018), who analyze the unemployment in France and Italy, respectively. The former studies the predictive capability of GT variables when forecasting the French unemployment in three age ranges 15–24, 25–49 and over 50, finding statistically significant results only for the youngest. The latter, supported by the selection bias argument, only focuses on the predictive capacity in youth unemployment. Additionally, Dilmaghani (2018, 2019) also investigates the effects of the digital divide on this methodology. Her first work analyzes the forecast improvements when introducing GT searches to predict the unemployment rate in the USA only for youths between 16 and 24 of age, distinguishing among Whites, Hispanics or African-Americans, and males or females. Her second paper shows an improvement in the prediction of the Canadian unemployment rate for the age grouped between 25 and 44 years old.

This section introduces the data employed in the analysis. We first describe the disaggregated (in gender and age) unemployment series. Second, we detail the GT queries that will be used as predictors devised to improve the forecast of the unemployment figures.

This section presents the models applied in the paper. First, we introduce the univariate models that will generate the benchmark predictions for each unemployment series defined by age group and gender. Second, we describe the alternative models that incorporate the information of the GT queries, which will potentially improve the benchmark forecasts.

This section analyzes the results of applying the previous models to forecast the Spanish unemployment by gender and age groups in an out-of-sample validation of 33 periods. However, the purpose of this section is not to merely predict the unemployment using Interned mined data. Instead, the main intention is to study whether the inclusion of GT predictors reveals information about a potential age and/or gender digital divide when forecasting the unemployment. Therefore, the forecasting performance is compared for Eqs. (2) and (3a–3b), which include GT data, across all the combinations of age groups and genders. For this comparison, the RMSE and the relative RMSE against the corresponding benchmark model are computed. All the forecasting models converge adequately and show no evidence of poor specification.

As the out-of-sample size is relatively small, there could be non-negligible uncertainty in the RMSEs. To incorporate this uncertainty in our evaluation of the predictive capacity, the forecast comparisons include the Diebold and Mariano (1995) test and its p value.⁷ The null hypothesis of this test is that the two predictions (coming from the benchmark and the alternative model) are equally accurate. Hence, a small p value evidences that the alternative model predicts better than the benchmark with a particular significance level.

We start by focusing on the comparison of different age groups with no gender disaggregation, denoted by All in tables and figures. First, Fig. 5 (top) shows the relative RMSEs against the benchmark’s for model (3a–3b), by age groups. Notice that the best predictions in terms of lowest RMSEs are usually found with a few GT variables. This is also revealed by the (low) values of J in Table 1 and seems to be consistent across genders and age groups. Second, both Fig. 5 (top) and Table 1 show no evidence of the youth bias found by Naccarato et al. (2018) and Fondeur and Karamé (2013) for Italy and France, respectively, as all the groups of age present a statistically significant benefit (around 10% level or less) of using GT searches as predictors. Third, contrary to the literature, the lowest forecasting gain when including Internet searches is found for the youngest age group, clearly rejecting an age digital divide effect in the total (male plus female) unemployed population.

Nonetheless, when those groups of age are disaggregated by gender, the results exhibit a completely different picture. First, the digital divide effect in favor of the youngest group is now clearly perceptible in men, as groups with age under 25 and under 45 are those with a higher forecasting improvement; see Fig. 5 (middle). This is confirmed in Table 2, where the gain in terms of RMSE is statistically significant at 10% only for these two groups. In fact, as the improvement for the group 25–45 is not significant, one could conclude that the benefit of using GT predictors for males’ unemployment only occurs for those under 25. Conversely, Fig. 5 (bottom) shows that GT searches systematically improve the forecasts obtained from the benchmark model for all the women age groups. When looking at Table 3, we observe that these improvements are indeed statistically significant at 5%. In fact, the benefit of including GT queries as predictors for the unemployment is much greater in females than males. Therefore, one can conclude that most of the gain found for the total unemployment series comes from the women side. This result has no precedence in the literature.

Digging into this finding, Table 4 and Fig. 6 emphasize the differences in the forecasting improvement, in terms of RMSE with respect to each benchmark, between women and men by age group. The gain is clearly greater for females in all groups of age. This differential seems not to be very relevant for the youngest group (1.8 percentage points), but it is remarkable for the rest of the groups (ranging from 18 to 31 p.p. of higher gain in favor of women, according to the age group; see Table 4). Besides, this result is independent of the number of predictors used to build the forecasts. Figure 6 shows that the women relative RMSE is consistently under the men’s one, no matter the age group or the number of predictors.

Beyond these main findings, it is interesting to analyze the queries that provide these results. Thus, we now discuss which are the best GT queries in terms of predictive power by gender and group of age. We will focus on two particular GT queries for each model: (1) the first predictor chosen by the feature selection algorithm, described in Sect. 4.2, and (2) the predictor that yields the highest forecasting gain in terms of RMSE reduction. Table 5 offers this information for men, women and total unemployment by age groups. When looking at male unemployment, there is not much variability in the best GT predictors found: either LinkedIn or Orange vacancies is the first feature selected by the algorithm or the best one, in terms of RMSE reduction for all age groups. When the prediction improvement is statistically significant (men younger than 25 or younger than 45), the only relevant GT query is LinkedIn, evidencing the importance of this social media networking site when it comes to searching a job, particularly for men. This is also noticeable in Fig. 5 (middle), where only one step-down is perceptible for men in most of the relative RMSEs.

Interestingly, this picture varies considerably when looking at female unemployment, where the variability of the best predictors is higher. Table 5 shows a diversity of terms (e.g., curriculum vitae, job vacancy, virtual employment office, job offers, etc.), suggesting that women are more likely to expand their job search to different queries, related to websites, firms and public institutions. Accordingly, Fig. 5 (bottom) shows several step-downs in women’s relative RMSEs. Contrary, none of the previous query terms seem to be informative enough to predict male unemployment. LinkedIn remains a very good predictor also for women unemployment in some age groups, which explains it remains the best predictor for most of the age groups when considering the total unemployment sample.

Last, to establish the generality of these findings, future research should analyze whether the results reported in this paper are robust to the phase of the business cycle in which the forecasts are computed. This does not seem obvious, as the unemployment series for women and men show more dissimilarities during the recoveries (and less in the recessions), except for the youths (see Fig. 3). This could partially explain the lower gender impact of the GT queries found for the unemployed younger than 25. Unfortunately, due to sample limitations, our forecast exercise only covers an unemployment recovery phase (shaded area in Fig. 3).

Some papers emphasize the idea that the Internet access is not yet universal even in advanced economies, and so this digital divide, either by age, gender or race, yields a selection bias in the use of data mined from GT, compared to other indicators. D’Amuri (2009), Fondeur and Karamé (2013) and Naccarato et al. (2018) find a digital divide effect in favor of the youngest unemployed (over the rest of ages), while Dilmaghani (2018) finds it in favor of Whites (over Hispanics and African-Americans) and (white) males over (white) females. The latter study only focuses on unemployed younger than 25 years of age.

Surprisingly, our research only finds an age digital divide effect in males unemployment. Its gain of 14.5%, in terms of RMSE, for the youngest unemployed is similar to the range 9-16% found by Fondeur and Karamé (2013), 15% by D’Amuri (2009), 15% by Vicente et al. (2015) and 10–19% by France and Shi (2018).

On the contrary, results on female and total unemployment suggest no evidence of any age digital divide effect. Further, GT queries turn out to perform much better as predictors for women unemployment than for men’s, for every group of age. To our knowledge, these results are unprecedented in the literature. When we examine these findings in contrast with the data supplied by the Spanish National Institute of Statistics (INE 2019), we observe that the digital gap by age was only significant in 2015 for people older than 45 years of age (a gap of 29% points of usual Internet use, with respect to the youngest people). In 2018, this gap had fallen to 13 p.p. When looking at the digital gap by gender—in the same survey—this has been closed from 2015 to 2018 for all working ages and even become slightly negative (in favor of women) for most of them. This could partially explain our results. However, we believe that the relation between GT variables and unemployment series is too much complex to be explained by just a digital divide effect, either by age or gender. Many other factors come into play here, specially when the measure of digital divide, as the difference of terms of usual Internet use, is small. For instance, the fact that women look up on search engines more than men (Gargallo-Castel et al. 2010), that the higher is the education level, the higher is the Internet use (Gómez 2019), and the country-specific structure of the labor market are likely additional variables of importance in order to explain the results obtained in this paper. In fact, van Dijk (2020) proposes that three main factors contribute to the digital divide: personal categories (e.g., age, gender, ethnicity, etc.), positional categories (e.g., labor, education, household, etc.) and resources. In this sense, only conclusions about the first factor can be drawn from this exercise.

Finally, future research should analyze how the results reported in this paper depend on the phase of the business cycle undergone during the validation sample.