Sending out an SOS: using start of rainy season indicators for market price forecasting to support famine early warning

Sub-Saharan Africa suffers from chronic food insecurity, and much of the region is experiencing increasingly shorter and more volatile rainy seasons (Wainwright et al 2019). Grain production and grain prices in the region are heavily influenced by local environmental conditions, especially rainfall (Lobell and Field 2007, Brown and Kshirsagar 2015, Davenport and Funk 2015, Davenport et al 2018, 2019, Funk et al 2018, Nobre et al 2018). At the same time, grain price forecasting is a critical component of food-security analysis and famine early warning (Rubas et al 2006, Vogel and O'Brien 2006, Jayne and Rashid 2010, Naylor and Falcon 2010, Funk et al 2019). For example, high wholesale grain prices prior to harvest can signal lower-than-average production, and high consumer grain prices can exacerbate food insecurity (Barret and Mutambatsere 2005). Many poor households spend 60% or more of their income on food, and higher food prices typically lead to reduced food access.

A relatively new tool to food-security analysis and famine early warning are analyses based around the start of the rainy season (hereafter, SOS) (Shukla et al 2021). Analyses based on SOS offer an intriguing advance to famine early warning, as SOS represents the first observable agronomic indicator in the season and starts a chain reaction of decisions, such as when to plant (Marteau et al 2011) ¹ , in-season resource allocation (Fink et al 2020), and when to harvest. These decisions can have direct impacts on local food availability. The SOS, both the amount of rainfall and when it arrives (relative to prior years), provides a key signal as to what the subsequent growing season and harvest will look like. In food-insecure areas dependent on regionally grown rainfed crops, SOS could potentially be a pivotal early indicator of food availability for the following year.

However, relatively little work has been done on the role that SOS indicators can play in forecasting. One recent paper (Shukla et al 2021) demonstrates the strong regional relationships between SOS and peak NDVI anomalies (an indicator frequently used as a proxy for grain production). Specifically, they find a large and statistically significant relationship in which late SOS is associated with decreased peak NDVI, indicating a relationship between late SOS and low plant productivity. This relationship is strongest in chronically food-insecure regions of east Africa. A recent conference presentation (currently a manuscript in-preparation) shows strong relationships between SOS and farmer planting decisions, indicating that SOS measures can be used to approximate planting dates when that data is unavailable (Krell et al 2019).

In this paper, we explore whether incorporating information on SOS ² can improve seasonal price forecast accuracy. We start by exploring a number of different SOS definitions using varying reference periods to define whether an SOS is 'on-time' or 'late'. We compare how those different definitions perform in seasonal price forecasting models.

Specifically, we examine if SOS indicators can predict maize price means over 6 and 9 month periods, or roughly the length of time from planting to market. We use different reference periods for defining 'early' versus 'late' seasonal starts based on the previous year's start date, or median start dates over 3, 5, and 10 years before the current season. In this way, we attempt to frame what the best reference time period is for determining if onset of the rainy season is late or not. We then compare the out-of-sample forecast performance of univariate time-series (autoregressive integrated moving average (ARIMA)) models with time-series (ARIMAX) models that include various SOS definitions as exogenous predictors (X).

We use these results to identify markets and regions where SOS measurements can enhance maize price forecasts. These markets present a typology that analysts can use to identify markets that are responsive to SOS conditions. While there is rich literature on the use of environmental variables to predict grain prices in developing countries (Brown et al 2006, 2008, 2012, de Beurs and Brown 2013, Algieri 2014, Brown and Kshirsagar 2015, Davenport and Funk 2015, Peri 2017), to our knowledge, this is the first paper to explicitly use and test rainfall-based SOS measures as a price predictor.

Our study focuses on several countries in eastern (Kenya, Somalia, and Ethiopia) and southern (Malawi and Mozambique) Africa. Figure 1 shows a map of the study area—the markets we examine and the approximate planting periods in the various regions. We focused our analysis on those markets in eastern and southern African countries with a minimum of 10 years' worth of maize (which is one of the major crops/food sources in this region) price data. The planting periods shown in figure 1 are from the GEOGLAM ³ Crop Calendars ⁴ and are approximations based on reviews of crop calendar data collected from international agencies, agricultural ministries, and expert knowledge (GEOGLAM 2021).

Zoom In Zoom Out Reset image size

2.1. SOS definition

2.2. Using varying reference periods to define anomalies

After calculating spatial averages for each year and month, we convert each of our predictor variables (SOS and 1st month's precipitation) into climate anomalies (value in a given month differenced from a median calculated from prior years) prior to model fitting. The use of anomalies as predictors is standard practice when working with environmental variables, especially in a climate context, as they place the key variable in a context relative to spatial and seasonal averages. In this paper, we explore if the reference period used to define an anomaly impacts model performance. Specifically, we define anomalies over a 1, 3, 5, and 10 year rolling median (rm) and compare the predictive performance of these anomalies across models. For example, if the performance of an SOS indicator based on a 1 year anomaly notably outperforms a model based on a 10 year anomaly, this provides useful information for future predictive models. Formally, the SOS anomalies are defined by:

$$\begin{equation*}\widetilde {{\text{SOS}_{\left( y \right)}}} = {\text{SOS}_{\left( y \right)}} - \text{median}\left( {{\text{SOS}_{\left( {y - 1:y - N} \right)}}} \right)\end{equation*}$$

where SOS is the dekad of season start in y and N refers to the 1, 3, 5, and 10 year reference periods.

As mentioned above, we also fit predictive models that use early season rainfall anomalies (instead of SOS anomalies) using rainfall means defined over the same 1, 3, 5, and 10 year periods. In the models that use rainfall anomalies, we only use the 1st month in the season, (defined by the crop calendars in shown in figure 1). We do this to compare two different but overlapping potential SOS indicators: when the season starts and how much rain falls at the very beginning of the season. Ultimately, this could serve to assist analysts in defining the best reference period and SOS metric for what makes a late versus an on-time SOS.

2.3. Price data

We use monthly wholesale and retail nominal ¹⁰ white maize prices for 74 markets in Ethiopia, Kenya, Somalia, Malawi, and Mozambique ¹¹ (figure 1). Data are collected, cleaned, and consolidated in the FEWS NET Data Warehouse, though not all data sets are publicly available at this time. All currencies were converted to USD per kilogram prior to model fitting. We only examine markets with ten or more years of continuous monthly price data. Since SOS events only happen once a season, we focus our analysis on seasonal price levels—specifically, monthly prices averaged over 6 and 9 month periods following approximate planting months (figure 1). This follows the goal of this paper in identifying how useful SOS indicators are in predicting seasonal price levels.

Figure 2 plots monthly mean maize prices (aggregated across markets) through the growing season. On the x-axis, '1' indicates approximate planting periods. For example, based on this figure, the planting period '1' might be March or April for eastern African countries and November for southern African countries. The colors indicate average prices (through the season) for years with a late versus an on-time season start. The figure suggests that late seasons tend to have higher prices, and these persist past the harvest periods. It should be noted that the highest levels of food insecurity often occur during the 'lean period', which falls within the rainy season, but before the harvest, i.e. around month three or four.

Zoom In Zoom Out Reset image size

Our baseline time-series model is a univariate (no exogenous predictors) ARIMA model,

$$\begin{align}y_{\left( t \right)}^{^{\prime}} & = {\phi _{\left( 1 \right)}}y_{\left( {t - 1} \right)}^{^{\prime}} + \cdots + {\phi _{\left( p \right)}}y_{\left( {t - p} \right)}^{^{\prime}} + {\theta _{\left( 1 \right)}}{\varepsilon _{\left( {t - 1} \right)}} + \cdots\nonumber\\ & \quad + {\theta _{\left( q \right)}}{\varepsilon _{\left( {t - q} \right)}} + {\varepsilon _{\left( t \right)}}\end{align} \tag{ 1 }$$

where ${y_t}$ is the time-series of observed prices averaged over either 6 or 9 months from planting, and the subscript t indexes years. The indicates potential differencing of the time-series, p is the order of lags, $\phi$ are the autoregressive parameters, q is the order of moving averages, ${\theta _{}}$ are the moving average parameters, and $\epsilon$ are forecast errors from the prior periods. ARIMA (p,d,q) models are standard and frequently used methods for time-series analysis and forecasting (Hyndman and Khandakar 2008, Hyndman and Athanasopoulos 2018) ¹² .

We investigate if SOS can be used to enhance forecast accuracy of seasonal grain prices. We compare accuracy of forecasted price means over 6 and 9 month periods. We analyze forecasts made with univariate ARIMA models (using just historical price means), ARIMAX where the exogenous variable is an SOS anomaly, and ARIMAX models where the exogenous variable is the 1st month's precipitation anomaly.

Our basic workflow is as follows. For each set of market time-series, we do the following:

• (a)  
  Fit univariate ARIMA (p,d,q) and ARIMAX (with an SOS measure as a predictor) models where the dependent variable is average price either 6 or 9 months from planting. Each market is initially fit on 5 years of data.
• (b)  
  Repeat this process for each market using an expanding window cross-validation approach and record the forecast error. For example, we fit a model on 5 years of data and forecast price averages in year 6. We then fit a model on 6 years of data, forecast year 7, and repeat. The final column in table 1 shows the total number of forecasts—for each of the 9 methods we evaluate—stratified by country and market type.
• (c)  
  The forecast errors for each series are averaged together to calculate one mean absolute percent error (MAPE) score for each market and method.
• (d)  
  Finally, we compare forecast error across markets, forecast horizons (6 and 9 month means), methods (with and without an SOS measure), and lags (1, 3, 5, and 10 years rm) used to define anomalies.

We evaluate forecast error using the MAPE. We analyze the MAPE in several ways. To provide an overview of performance, we use Nemenyi/MCB (multiple comparison with best) tests (Koning et al 2005, Demšar 2006). These tests were pioneered to perform a post-hoc analysis of the methods used in the M3 ¹³ forecasting competition. The basic process is to rank the forecast performances of each method on each time-series, and then estimate median ranks of performance along with confidence intervals for those ranks. Intuitively, this is similar to doing a regression on the ranks of the MAPE scores using dummy variables for each modeling approach and then comparing the estimated coefficients for each model.

However, we do not want to rest our analysis solely on significance tests (Wasserstein and Lazar 2016, Amrhein et al 2019, Wasserstein et al 2019). We are more interested in regional variation in the value of SOS predictors and identifying areas where SOS indicators add value to food-security analysis. Therefore, we also present several descriptive measures to summarize model performance. We plot the performance of each SOS-based model, relative to the univariate model, across countries (and season types) to identify any emerging patterns. Finally, we map the markets where SOS-based indicators consistently outperform univariate models, regardless of the indicator used.

4.1. Nemenyi tests comparing ranks of model accuracy scores

We first present an overall summary of how the different models performed. Figure 3 shows estimated MAPE ranks and 90% confidence intervals resulting from the Nemenyi tests (nine methods evaluated across 74 markets for a total of 666 MAPE scores). For the 6 months time period, we see that the univariate model, on average, performs worse than all others, and adding SOS or 1st month's precipitation anomaly as a predictor results in higher forecast skill. In the 9 months period, the univariate ARIMA model is 2nd to last, outperforming the 5 years lag-based SOS model. The models based on 1 and 10 year anomalies all perform best in either period. We suspect the increased performance of SOS in the 6 months forecast period is due to the fact that, as observations get farther from the SOS, more information and events outside of initial conditions could start influencing prices, diminishing the importance of initial conditions. In the middle of the growing season, grain from the previous season is being consumed, food availability is low, and prices tend to increase (figure 2). A poor start to the season may contribute to a perceived increase in risk and food prices.

Zoom In Zoom Out Reset image size

4.2. Results by country

Figure 4 shows results by country (aggregated across market). The x-axis shows the difference in MAPE scores between SOS/precipitation-based models and the univariate ARIMA model. Points that fall to the left of the vertical line indicate that a given model outperformed the univariate model. Colors indicate whether the prediction was made in a normal/early year versus a late start year (for a given market area).

Zoom In Zoom Out Reset image size

In general, we see the most support for SOS indicators improving forecast performance in Ethiopia and Kenya, with the strongest results for the 1 and 10 years based anomalies and, as expected, stronger performance when the season start is defined as late. In Somalia, these same results hold for the shorter periods (1, 3, or 5 years), but not for the 10 year period. In contrast, results are more ambiguous for Malawi and Mozambique, where the indicators often do not outperform the basic model even in late years. In general, we do not see large performance differences between the different types of SOS indicators (SOS anomaly or the 1st month's precipitation), although the SOS anomaly-based model has consistently shown the best performance in Ethiopia during the 6 months forecast window.

4.3. Results by market

Finally, we present results at the market level. Figure 5 shows markets where price models with SOS indicators beat the univariate models by more than 5% (regardless of lag type or indicator used). The purpose here is to identify markets where some form of SOS indicator can improve early season price forecast performance. Shapes indicate market type (wholesale versus retail), and darker colors indicate the strength of the SOS-based ARIMA model against the univariate model. Grey points are markets where SOS indicators do not increase forecast performance. In general, we see that SOS indicators primarily improve forecast performance in central Ethiopia, southern Kenya, and southern Somalia. With some exceptions, the spatial distribution of the successful SOS models aligns with drier, more drought-prone eastern areas (see supplemental figure S2). These regions have shorter seasons and less opportunity to recover from a delayed onset. There are only a few markets in Malawi and Mozambique, where SOS indicators routinely increase forecast performance.

Zoom In Zoom Out Reset image size

We find that using some form of SOS indicator (either an SOS anomaly or 1st month's rainfall anomaly) leads to increased predictive power when examining prices over a 6 months window. However, the results vary considerably by country. We find the strongest performance of SOS indicators in central Ethiopia, southern Kenya, and southern Somalia. We find less evidence for the use of SOS indicators for price forecasting in Malawi and Mozambique. These findings support other research suggesting that, within sub-Saharan Africa, the relationship with SOS and food-security measures is strongest in east African countries (Shukla et al 2021).

The evidence demonstrating that using a specific reference period (1, 3, 5, or 10 years) to define a season anomaly will increase performance is less conclusive. While the MCB tests suggest that all anomaly definitions tend to outperform univariate models, there is not an obvious pattern, as both the 1 and 10 years based anomalies were ranked as the overall best indicators, and there was substantial heterogeneity of these results across regions. One explanation for the performance of the 1 and 10 year anomalies is that a late start or low rainfall based on a 10 years anomaly may signify a substantial drought, while the 1 year anomaly simply reflects a cognitive bias towards the most recent events (short-term memory).

The relative insensitivity of the reference period may be also related to the year-to-year stationarity of the African monsoon systems. These systems move south and north in response to latitudinal variations in insolation and shifts in the Indian Ocean's complex annual wind patterns (Funk et al 2016). Given this strong climatological cycle, the baseline 1, 3, 5, and 10 year values used to calculate the anomalies could be quite similar.

This same feature—the strong phase locking of the seasonal cycle, can also help to explain the value of SOS indicators. In drier areas of central-eastern East Africa, the end of the rainy season is quite fixed, so a slow onset is an effective harbinger of higher prices. In Malawi and central-northern Mozambique, cessation of the monsoon season is less distinct, and this may help account for the weaker SOS relationships.

We also suspect that the use of historical references to define what is a 'normal' season likely varies significantly across market-sheds, buyers, sellers, and even farmers; this heterogeneity creates a more muddled picture when aggregated across countries and years. For example, the 10 year indicators did not perform as well in Somalia, which may reflect the long period of civil conflict in that country (e.g. other factors, such as conflict, pests, and disease can inhibit the influence of long-term seasonal anomalies on market prices). It is possible that this heterogeneity in response to seasonal trends, along with uneven patterns of spatial price transmission in the region (Brown et al 2012, Ansah et al 2014), also explains why the results are less spatially clustered than we would expect ¹⁴ . A final source of uncertainty is the lack of explicit knowledge of planting dates. While we have preliminary evidence that planting dates generally coincide with SOS (Krell et al 2019), and we have general knowledge of regional crop calendars (GEOGLAM 2021), we also know that the timing of planting is driven by a host of factors and that these may vary by year across regions. This is one of the reasons why we focus on aggregate price means and limit our results to markets with at least 10 years' worth of price data—so that our conclusions focus on price levels across seasons and regions rather than a specific market, month, or year. We anticipate that the heterogeneity of individual farmer's or market's responses to SOS could be better teased out with future survey data or field experiments. However, we are confident in asserting that, when making forecasts, the specific reference value is less important than simply including some measure of early season anomaly, particularly in east African countries.

This has important potential applications in terms of price forecasting and intervention planning. Our results imply that late onset of the rainy season can impact market behavior quickly, months before harvest (figure 2). If price variations are driven by expectations, rather than actual physical grain deficits, then that can help explain the rapid, and potentially deadly, jump in prices seen in Kenya, Somalia, and Ethiopia during the drought years 2011 and 2017 (Funk et al 2018) ¹⁵ . These price spikes occurred in the middle of the growing season, long before harvest. These rapid and dangerous price shocks require rapid and effective responses, which can be supported by staged early warning systems (Funk et al 2018).

In regards to future research opportunities, there are several intriguing avenues. The first might be to investigate explicit monthly price movements, not seasonal averages, as we do here. Our goal in this paper is to provide a general sense of the usefulness of SOS indicators; however, as the results suggest, there is considerable heterogeneity in market price behavior, and thus, a more finely detailed exploration at the monthly and individual market level might provide more insight as to why certain markets do or do not respond to seasonal start signals. A finer-grained analysis might also explore alternate crops, a wider range of forecast windows, and a more detailed examination of wholesale and retail level prices. We are also interested in exploring SOS definitions with finer temporal resolution (perhaps at the daily level) to examine the extent that this might influence very near-term price movements and volatility. Finally, there is an emerging set of Earth observation products that can be used to forecast SOS which, if quantitatively linked to food-security outcomes, could also be promising for developing extended seasonal outlooks.

An early version of this paper was presented (as a poster) at the 2019 AGU meetings in San Francisco. Juliet Way-Henthorne served as an in-house editor for several versions of this paper. Our sincere thank you to Sonja Perakis and the FEWS NET data and price teams for their assistance with obtaining and understanding the data and for their insightful feedback on numerous versions this manuscript. Funding for this research came from several sources. Including USAID (Award # 72DFFP19CA00001), the NASA SERVIR program (Award # 80NSSC20K0159), and the DAPRA World Modelers program (Award # W911NF-18-1-0018). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the position or the policy of the Government or the Prime Contract and no such official endorsement by either should be inferred.

Finally we thank two anonymous reviewers for their time and critical feedback.

The data generated and/or analyzed during the current study are not publicly available for legal/ethical reasons but are available from the corresponding author on reasonable request.