Interpretation of comparative LCAs: external normalization and a method of mutual differences

Research in life cycle assessment (LCA) methods focus predominantly on building life cycle inventory (LCI) databases (Frischknecht et al. 2004; Suh and Huppes 2005; Miller and Theis 2006; Dones et al. 2007; Verbeeck and Hens 2010; Jungbluth et al. 2012), calculating new midpoint characterization factors (Koellner and Scholz 2007; Pfister et al. 2009; Van Zelm et al. 2009; Gallego et al. 2010; Saad et al. 2011; Hauschild et al. 2013), and improving end-point damage modeling (Hayashi et al. 2006; Boulay et al. 2011; Motoshita et al. 2014). Less emphasis has been placed on normalization and weighting, which are optional practices in life cycle impact assessment (LCIA).

Nonetheless, normalization and weighting remain crucial in comparative LCA to provide decision support in the face of uncertain environmental tradeoffs—such as when one alternative performs best in some areas and worse in others. In problems of comparative technology assessment, characterized results alone seldom result in a definitive environmental choice, leaving decision-makers to confront complex environmental tradeoffs largely unaided in examples critical to sustainability (Rowley et al. 2012). These environmental tradeoffs exist regardless of the completeness of characterization factors or LCI databases. Thus, there is a critical need for analogous research efforts focused on normalization and weighting as tools to improve decision support in LCA.

There are numerous impact assessment methods available to LCA analysts that apply to any LCI, each with their own external normalization references. These tools provide alternative methods of characterizing and interpreting environmental impacts from the myriad and disparate chemical releases reported in a LCI. Some of the most widely applied LCIA methods are as follows:

According to ISO standards, normalization provides context to characterized results by relating results to a reference information (ISO 2006). Typically, as is the case with the main LCIA methods, these normalization references represent a community such as Europe, USA, or the world. The reference information can also represent another system, or an internal baseline such as the largest or smallest alternative. There are two distinct approaches to normalization known as external (when the reference is outside the study) or internal (when the reference is within the study). While both approaches are mentioned in the ISO guidelines, external normalization approaches dominate current practice, with the exception of the comparative bar charts typically produced by LCA software packages that scale results via division by a maximum.

This paper presents a methodological evaluation of life cycle impact assessment (LCIA) that isolates the effect of normalization across multiple LCIA methods and comparative LCA applications. Three current practices, CML EU25 2006 and World 2000 (Sleeswijk et al. 2008), ReCiPe midpoint H version 1.10 European and World (Goedkoop et al. 2009), and TRACI US 2008 (Bare 2011; Ryberg et al. 2014) are compared to the overlap area approach in application to four comparative LCA studies, and patterns in the results are examined for evidence of systematic biases. The findings help inform LCA practitioners of the implications in the choice of normalization methods in comparative assessments.

Results show the systematic effects of external normalization and the overlap area in ReCiPe, CML, and TRACI. Figure 2 highlights the impact categories most influential in each LCIA across all four comparative LCA applications. The x-axis represents the impact categories within each LCIA method organized according to the coefficient of variation so that the impact categories to the right have the largest uncertainty. The y-axis shows the contribution according to Eq. (3). Depending on the LCIA, there can be two series in the graphs, corresponding to a European and global normalization reference.

Within ReCiPe H Midpoint, EU and World normalization references highlight toxicity and eutrophication-related impact categories with the exception of natural land transformation, which is larger when using the EU reference. Impact categories most salient in EU and World references replicate across the four LCA applications indicating the possibility of a systematic bias in the normalization approach. With the exception of Kasah (2014), this same pattern in ReCiPe externally normalized results was found in recent LCA applications such as concentrated solar power (Corona et al. 2014), structural beams (Ibbotson and Kara 2013), industrial cleaning products (Kapur et al. 2012), diapers (Mirabella et al. 2013), laundry detergents (Prado-Lopez et al. 2014), energy recovery from rice husks (Prasara-A and Grant 2011), and dishwashing soap (Van Hoof et al. 2013).

Normalization references within CML baseline, EU 25 and World 2000, reveal that a single impact category, marine aquatic ecotoxicity, as the most and only dominant impact category (Fig. 2). Here, the selection of alternatives in each example (PV, eGrid, concrete, and paper pulp) is determined by single impact category. A survey of recent publications from the International Journal of Life Cycle Assessment reporting normalized impacts from CML supports the finding that marine aquatic ecotoxicity is the most influential category in evaluations of water services (Barjoveanu et al. 2014), diapers (Mirabella et al. 2013), exterior household walls (Monteiro and Freire 2012), packing tape (Navajas et al. 2014), thermal insulation (Struhala et al. 2014), and an entire inventory data library (White and Carty 2010). These results were also reproduced by Sim et al. (2007) in a food sourcing application where authors excluded marine aquatic ecotoxicity from normalized results due to masking of other aspects. Alternatively, some exceptions can be found in soil remediation processes (Busset et al. 2012) and pad-dyeing technology (Yuan et al. 2013).

The US 2008 normalization reference within TRACI highlights carcinogenics, non-carcinogenics, and ecotoxicity impact categories (Fig. 2). These externally normalized results are also replicated by Rostkowski et al. (2012) in a bioplastic example and again by White and Carty (2010) in 800 different processes.

Alternatively, application of the overlap area to the four comparative LCA application is shown in Fig. 3. The x-axis represents the impact categories as characterized by ReCiPe, CML, and TRACI arranged according to the coefficient of variation. The y-axis measures the tradeoff significance, Ψ _i , in each application ranging from 0 to 1 according to Eq. (4). For example, in the PV comparative LCA application (individual graph in the top left corner of Fig. 3), the performances at characterization of alternatives (CdTe, A-Si, Single-Si, Multi_Si, and Ribbon-Si) are most different in water depletion (high Ψ) and nearly undistinguishable in freshwater and marine ecotoxicity (low Ψ) when using ReCiPe characterization factors. For each application, the overlap area calls attention to the most significant differences, the tradeoffs, to guide the selection process. Unlike in external normalization, the overlap area results show no clear pattern across applications. More importantly, the relative scales of tradeoff significance across impact categories lies within the same order of magnitude. Thus, avoiding an assessment where a few or a single impact category dominates results.

The normalization references evaluated in this study show a systematic bias where the same impact categories are highlighted regardless of the LCA application, uncertainty or geographical coverage of the normalization reference. In all four applications, the use of global or regional normalization references had little influence in the outcome of the results as both point out to the same impact categories. Therefore, previous recommendations on the choice of a normalization reference fail to avoid bias. Within ReCiPe H midpoint, the highest normalized contributions in the EU and World normalization references come from freshwater ecotoxicity, marine ecotoxicity, and human toxicity (Fig. 2). Similarly, both EU25 and World 2000 normalization references within CML baseline consistently identify marine aquatic ecotoxicity as the largest contributor to normalized scores across all four applications (Fig. 2). Finally, TRACI US 2008 normalization reference identifies carcinogenics, ecotoxicity, and non-carcinogenics the most influential impact categories in all four comparative LCA applications (Fig. 2). Consistently, the external normalization approaches in the three LCIA methods highlight toxicity-related impacts. This may be due to the underestimation of toxicity impacts in the area of coverage in the normalization reference dataset because of the lack of emission data and characterization factors. Previous studies comparing LCIA methods shed light on gaps in characterization factors and explain how that may influence the outcome of results (Huijbregts et al. 2003; Cavalett et al. 2012; Castellani et al. 2016). For instance, the issue of characterization of impacts of metals has been an issue in impact assessment that leads to an overestimation of ecotoxicity impacts as seen in CML-IA (Ligthart et al. 2004; Heijungs et al. 2007). Given the consistency of results over diverse applications, it is unlikely this bias is due to an overestimation at inventory level in the product system (as newer inventory libraries become available, this type of evaluation could help point out biases early on). These and other conceptual issues in normalization have been discussed extensively in the literature (Bare and Gloria 2006; Bare et al. 2006; Sleeswijk et al. 2008; Finnveden et al. 2009; Bare 2009), but what is most concerning is that these gaps can have such strong influence on the results. It is also concerning that the results obtained in this study are shown repeatedly in the recent literature across a wide range of applications with different temporal and geographical scales including water systems, steel beams, cleaning products, and bioenergy.

It is important to have these limitations in mind when interpreting externally normalized results in improvement assessment. One recommendation is to include hotspot analysis from outside of LCA and call attention to flows not characterized to complement the analysis and obtain more robust conclusions (Castellani et al. 2016). Solving issues in data completion requires extensive and constant data compilation efforts to achieve a complete and up-to-date normalization reference (Kim et al. 2013). These efforts, although theoretically plausible, suffer from great practical challenges as achieving data completion is an endless, laborious task. Ideally, a perfect normalization reference will not have such biases but rather show a more balanced result that varies across inventories. The questions are then, “How will we know when we have an adequate normalization reference? How can we obtain unbiased results?” Previous studies have arbitrarily attempted to scale down normalized data “peaks” as in White and Carty (2010), but such repair efforts lack a fundamental basis.

Alternatively, the use of planetary boundaries or carrying capacity based normalization references has been proposed as a way to provide a better measure of absolute impacts (Bjørn et al. 2015; Bjørn and Hauschild 2015; Fang et al. 2015). This can function as some sort of “threshold” that rejects alternatives exceeding such planetary boundaries following a strong sustainability approach (Janeiro and Patel 2015). The use of the planetary boundaries can also help prioritize areas for improvement and set reduction goals (Sandin et al. 2015). However, there are some issues remaining with planetary boundaries that hinder its application in LCA. For instance, reaching consensus over the planetary boundary per impact category and scaling to a relevant geographical and/or product scale. It is also unclear how to deal with impact categories that exceed established planetary boundaries. With advancements in these areas, planetary boundaries could serve as a basis for narrowing down alternatives and complementing existing interpretation approaches for comparative and improvement assessment.

In contrast to improvement assessment, external normalization remains wholly unsatisfactory for the purposes of comparative LCAs, even in the case of complete datasets. The issues highlighted in the paper concern the practice of external normalization in any impact assessment beyond the illustrated three LCIA methods. While more up-to-date inventories, characterization factors and external normalization reference datasets may produce slightly different results (Owsianiak et al. 2014), the fact is that when evaluating competing alternatives, any form of external normalization will by definition fail to evaluate mutual differences. So, while there are more up-to-date LCIA methods available (such as the ILCD method), the interpretation approach for comparative LCAs remains limited. Thus, the problem identified in this paper goes beyond data completion solutions and rather it calls for a paradigm shift in the way we approach interpretation of comparative LCAs.

Even in the case of an ideal normalization reference, external normalization does not evaluate mutual differences with respect to uncertainty. Thus, it does not highlight the tradeoffs in a particular decision and changes in information that may increase or decrease uncertainty ranges will not influence the results. Currently, the level of uncertainty that is propagated up to characterization does not determine whether an impact category is highlighted or not. For instance, the normalization references in ReCiPe and TRACI highlight the impact categories with the largest coefficient of variation (Fig. 2), so they can lead to recommendations driven by the aspects of largest uncertainties. In addition, in the case of preparing for weighting, external normalization applies a linear function that allows for a single, or few, impact categories to dominate the assessment, as seen in previous cases of weight insensitivity and studies describing compensation in aggregation methods (Rowley et al. 2012).

Instead, this study calls for decision-driven (Stern and Fineberg 1996) approaches that evaluate mutual differences and incorporate data uncertainty in the interpretation of comparative results. As shown here, the overlap area uses quantitative uncertainty information, which can derive from inventory, characterization, and/or methodological choices (Mendoza Beltran et al. 2016). We can identify relevant aspects of a comparison with the overlap area between probability distributions or other approaches that evaluate mutual differences (Heijungs and Kleijn 2001; Prado-Lopez et al. 2014; Henriksson et al. 2015; Gregory et al. 2016; Wei et al. 2016). These results do not suffer from the systematic biases of external normalization, as the outputs vary for each LCA application (Fig. 3). Unlike external normalization, overlap area results adapt to changes in uncertainty information and can guide the decision towards those aspects where we have the most knowledge. Aspects with larger uncertainties that make alternatives indistinguishable from each other can then be identified as areas that benefit the most from data refinement efforts (Prado-Lopez et al. 2016).

The overlap area does not serve as a basis for weighting but it does help illustrate key properties required in alternative methods of aggregation. Previously, outranking has been proposed as an alternative method of normalization that can serve as preparation for weighting (Cinelli et al. 2014). Along with weighting, it allows aggregation of to an overall (probabilistic) score as shown in Prado-Lopez et al. (2014) and Rogers and Seager (2009). Outranking evaluates pairwise mutual differences with respect to uncertainty data of characterized results and applies a nonlinear function that avoids full compensation between good and poor performances. Compared to external normalization, outranking generates results that are more sensitive to different weight ranges (Rogers and Seager 2009).

This study shows that the effects of external normalization may overwhelm differences in inventory and technology applications. In ReCiPe, CML, and TRACI, external normalization highlights the same set of impact categories across four diverse representative applications even if they utilize distinct inventories. The same results were found in multiple other studies in the literature, thus providing further evidence of these systematic biases. As a consequence, results of external normalization can lead to recommendations based entirely in the normalization approach. Therefore, when using external normalization references for hotspot identification, it should be done with care and it may be useful to perform some complementary analysis. However, when dealing with multiple alternatives and tradeoffs between them, as in a comparative LCA, these systematic biases are an issue that is not resolved by better data or a different geographical coverage area (as both regional and global references had biases). External normalization is not suitable for comparative LCA, as the normalization step imposes a linear aggregation among impact categories that can dominate most weight schemes. In fact, any type of division by an external reference, whereas it pertains to total per capita impacts of a community or a particular aspect such as average per capita nutrition, mobility or energy use, will by definition fail to evaluate mutual differences and integrate uncertainty in the results. Rather, comparative LCA calls for a decision-driven approach for evaluation of relevant differences that is consistent with internal normalization.

Decision support in a comparative LCA must focus on evaluating the impact of the decision (e.g., elucidating tradeoffs) rather than the impact of individual alternatives (such as hotspot identification). This is achieved by evaluating mutual differences relative to uncertainty—a practice that has been adopted in several approaches. This practice is sensitive to changes in uncertainty of information, frees studies from issues of data completion in normalization references, enables prioritization of data refinement, and is sensitive to stakeholder and decision-maker values. This study illustrates the overlap area as one of the approaches to identify relevant aspects of a decision and proposes a paradigm shift in the way we approach interpretation of comparative LCAs.