Measuring Natural Hazard-Related Disasters through Self-Reports

Climate change is leading to an increased number of natural hazards and disasters in many countries (Oppenheimer and Anttila-Hughes 2016). While all members of a community can be negatively affected by natural hazards and disasters, children are often disproportionately affected (Masten and Narayan 2012; Codreanu et al. 2014; Kousky 2016; Dyregrov et al. 2018). To date, research on the impacts of natural hazard-related disasters on children has largely focused on physical and mental health, and schooling (Kousky 2016).

Although a number of studies have found the impact of natural hazard-related disasters on children’s well-being to be small, some studies have found significant long-lasting impacts. For example, a study of the long-term impacts of the Australian Black Saturday bushfires in 2009, which resulted in the loss of 180 lives, found that children in primary school who had been in bushfire-exposed areas performed substantially worse on numeracy and literacy tests, and this effect continued four years after the fires (Gibbs et al. 2019). Similarly studies of the impacts of major Australian bushfires in 1983 found higher levels of psychological symptoms among children two years after the fires (McFarlane 1987) and some lingering effects after 20 years (McFarlane and Van Hooff 2009). There is also some evidence of impacts in utero. For example, a month after Super Typhoon Haiyan hit the Philippines in 2013, a qualitative study revealed that pregnant and lactating women were among the most vulnerable healthwise in the aftermath (USC-OPS 2014), indicating an impact that is likely to extend to offspring in their formative stages. Most studies of the impact of natural hazards and disasters on children have focused on particular areas and on a particular disaster. Kousky (2016) noted that there is a need for research to examine the cumulative effects of experiencing multiple natural hazard-related disasters on children living in high-risk disaster areas.

Although there are well-established scientific methods for quantifying the occurrence and impact of a number of types of natural hazard-related disasters, methods for quantifying other disasters, such as drought, are not well established. Moreover, we are not aware of a standard technique for quantifying exposure to multiple natural hazard-related disasters. There are also challenges related to quantifying the impact of natural hazard-related disasters on human populations. Developing measures of disaster impact is important because the measures can potentially be used to inform the development and implementation of disaster risk reduction strategies that could reduce the impact of exposure to natural hazards and disasters (World Bank and United Nations 2010).

The Philippines is an ideal country in which to study the impact of natural hazard-related disasters on human development because it has a large number of such disasters. In 2018, 6.5 million people were affected by natural hazard-related disasters, the second highest number after India (data from the International Disaster Database – EM-DAT, CRED 2017). Moreover, recent research in the Philippines reported that rainfall shocks are associated with chronic total poverty and food poverty (Bayudan-Dacuycuy and Baje 2019). In this article, we report on the development and validation of a measure of natural hazard-related disasters based on self-reports of exposure to disasters collected as part of a national cohort survey of children in the Philippines, the Longitudinal Cohort Study on the Filipino Child (LCSFC) (USC-OPS 2018). The self-reports of exposure to natural hazard-related disasters were obtained from the primary caregivers of the study children.

There are a large number of studies that have used self-report measures of exposure to disasters (Verger et al. 2003; Garfin et al. 2014; Harville et al. 2015; Sherwood et al. 2017), but they have not combined individual-level responses to produce an area-level measure. The LCSFC also collected unusually rich data on disaster exposure and impacts reported by caregivers and barangay (smallest administrative unit in the country) officials. In addition to these measures, information compiled in EM-DAT (CRED 2017) is linked at the barangay level to validate the survey measures of disasters that were constructed from individual-level self-reports.

In this study we used self-reports of natural hazard-related disasters and then aggregated them to the geographic area level (Hunter et al. 2012; Edwards et al. 2015, 2019). Aggregation of area ratings by self-report to neighborhoods is not new. It has been routinely used since the late 1990s in studies examining the physical and social environments of neighborhoods (Sampson et al. 1997; Mair et al. 2010). The approach is widely used and accepted. An article published in Science describing the technique has been cited by other papers more than 12,000 times, suggesting that the method has been acknowledged by the research community as legitimate for the construction of area measures (Sampson et al. 1997).

The main advantage of a measure of exposure to disaster based on individual self-reports is that the impacts of disasters are often geographically localized. Flooding is a good example of this type of disaster—there could be extensive damage to houses in low-lying areas but not to houses that were built on higher ground in the same village. Thus individual self-report is likely to more accurately measure a household’s exposure to disasters than measures derived from national or international monitoring systems such as the EM-DAT, or reported at the community level by officials. It is therefore likely to be a better measure if the aim of a study is to estimate the impact of experiencing a natural hazard-related disaster on household-level outcomes.

One limitation of self-reported measures of disasters is that they can be influenced by individual circumstances, such as exposure to poverty or mental health issues. We overcame this limitation by creating an individual-specific average rating of disaster exposure in the local area that uses neighbor reports and excludes caregivers’ own reports in constructing the measure for caregiver households. In this way the measure of neighborhood disaster exposure is exogenous to individual circumstances but retains the capacity to capture geographic variation in disaster exposure.

The study also developed a measure of the impact of natural hazard-related disasters in which impact is conceptualized in terms of economic or health effects. This disaster impacts measure is only for the household level, since many household-specific factors can mitigate severe impacts of natural hazards and disasters—for example, precise location of housing, quality of housing, and economic reserves held by a household. The measure of disaster impacts is nevertheless based on reports on objective characteristics such as damage to houses or property, loss of livelihood, or injuries to household members.

The general aim of this study was to develop measures of disaster exposure and impact from individual self-reports. We also aimed to establish their convergent validity with objective disaster measures—official reports and the International Disaster Database—and their predictive validity with household income and income adequacy. Although there was limited research literature to generate specific hypotheses, we had some more specific aims: (1) to test whether individual self-reports of disaster exposure were significantly associated with objective disaster measures (convergent validity); (2) to examine whether there was sufficient community variation in self-report measures of disaster exposure at the community level and ecometric reliability to support average ratings of other residents at the community level (average community report); (3) to test whether average community reports and individual reports of disasters were correlated with objective disaster measures to test convergent validity; (4) to construct an individual measure of impacts of natural hazard-related disasters using factor analysis—we tested whether this measure of disaster impacts was correlated to other disaster measures (evidence of convergent validity) but not too highly (evidence of discriminant validity). Finally, given that it is well documented that there are household economic impacts of natural hazard-related disasters (Anttila-Hughes and Hsiang 2013; Botzen et al. 2019), we examined whether our disaster measures were associated with income and income adequacy (evidence of predictive validity).

This section provides an overview of the data sources and statistical methods used in this study.

The first section validates the self-reported experience of disasters. The second section reports on the reliability of community reports of disasters and the third section on the convergent validity of self and community reports of disasters. The fourth section reports on the development of a variable measuring disaster impacts and the fifth section focusses on the validity of the measure of disaster impacts. The final section examines the relationship between disaster measures and household economic outcomes.

This article reports on the development and validation of cumulative measures of exposure to natural hazard-related disasters, at the individual household and community levels, using Philippines data for the period 2013–2017. It also reports on the development of an individual household-level measure of the impact of natural hazard-related disasters. First, we showed that the individually reported household cumulative exposure had statistically significant associations with disasters reported by officials responsible for the geographic area and with disasters collected in EM-DAT. Also noteworthy was that 25% of variation in individual reports of exposure occurred at the community level, and that average community reports demonstrated good ecometric reliability. We then generated a community-level measure of exposure to natural hazard-related disasters based on neighbors’ reports, but not individual self-reports, thereby providing an exogenous measure of disaster exposure in the local area for each household.

Second, we showed that these community-level scores are more strongly related to EM-DAT and reports from barangay officials than individual reports. Third, we developed a summary score of disaster impacts at the individual level that has adequate internal consistency. The rationale for an individual-level report of disaster impacts was that, whereas exposure is likely to be a shared experience, many household factors will reduce the impact of exposure to a disaster (for example, quality housing, sources of livelihood). Importantly, this individual-level measure of disaster impact was associated with measures of exposure (individual and area level), barangay official community-level reports, and EM-DAT (CRED 2017). However, the impact measure was only moderately associated with the community average, indicating that it was an independent measure of disaster impact.

One key outcome of a natural hazard-related disaster is the economic impact on households and communities (World Bank and UN 2010). We showed that our preferred measure of disaster exposure, the average community report, and disaster impact show evidence of predictive validity in that they are consistently related to household income and the adequacy of income in households.

Previous research in this area is limited, but our findings do align with the existing research. For example, Hunter and colleagues (2012) showed that a self-reported measure of drought was related to objective measures of rainfall, that there was substantial agreement within geographic areas on whether there is a drought, that the heterogeneity of farmers’ reports of drought was due to access to other ways of sourcing water through irrigation, and that self-report measures of drought were correlated with financial hardship and worsening financial position. In subsequent work, the same research group showed that self-reported drought, aggregated at the area level in the same way as in our study, was associated with mental health problems, financial stress, decreases in household income, and loss of services (with those engaged in agriculture experiencing greater impacts on mental health, financial stress, and household income) (Edwards et al. 2019).

One of the limitations of the current study was that the measures of disasters varied in the time period that was covered. Whereas household ratings reported on the past 3 years and barangay official community-level reports covered the previous 3 years, the cumulative data captured in EM-DAT was for the previous 11 years. Therefore, it is not surprising that the relationship between the variables was not higher. Coding of the EM-DAT data is under way to establish a temporally consistent variable. Another limitation of self-reported measures of disasters is that they can be endogenous, or influenced by individual circumstances such as exposure to poverty or mental health issues. We overcame this limitation by creating an individual-specific average rating of disaster exposure in the barangay that uses neighbor reports but not caregiver reports. The fact that the two other “objective” measures of disasters—reports by barangay officials and EM-DAT—were more highly correlated with community average reports reinforces the strength of this methodological approach.

We chose to develop an individual measure of disaster impact, but EM-DAT also has estimates of the economic impacts of each natural hazard-related disaster. A limitation of EM-DAT data for the purposes of this study is that these economic impacts are estimated for the whole disaster and not disaggregated to small neighborhood areas. Although we did have another source for impacts—reports from barangay officials—these reports were subject to substantially missing information (40% or more barangays). Therefore, although there are attractions in using another rater of disaster impact or another source of information on disaster impact, the significant data limitations precluded such an approach in this study.

This research has several implications. The study further demonstrates that individual self-reports, when aggregated at the area level, can be a valid and reliable measure of disaster exposure (Hunter et al. 2012). This has direct implications for researchers. In the same way that “ecometric” measures of neighborhood social capital and disorder (Sampson et al. 1997; Raudenbush and Sampson 1999) heralded an explosion of research into neighborhood-level measures of social capital and physical disorder, there is potential for our methodology to be adopted by other research groups to generate more nuanced measures of people’s disaster exposure than are currently available from objective measures such as EM-DAT. Our technique could also be applied beyond surveys to crowd-sourced information through social media or other online platforms—with further development and refinement, these could be used as “social barometers” of the impact of extreme weather events on populations. They could potentially be used to monitor the implications of climate change, and lead to better and more nuanced support and mitigation strategies. Beyond improvements and refinements of measures of extreme weather events, we confirm prior research on the economic impacts of natural hazard-related disasters on households (Anttila-Hughes and Hsiang 2013; Botzen et al. 2019).