Life cycle assessment (LCA) is intended as a quantitative decision support tool. However, the large amount of uncertainty characteristic of LCA studies reduces confidence in results. To date, little research has been reported regarding the comparative sources of uncertainty (and their relative importance) and how, or how commonly, they are quantified in attributional and consequential LCA. This paper answers these questions based on a review of recent LCA studies and methods papers, and advances recommendations for improved practice.
Methods
All relevant LCA methods papers as well as case studies (amounting to 2687 journal articles) published from 2014 to 2018 in the top seven journals publishing LCA studies were reviewed. Common sources and methods for analysis of uncertainty in both attributional and consequential LCA were described, and their frequency of application evaluated. Observed practices were compared to best practice recommendations from methods papers, and additional recommendations were advanced.
Results and discussion
Less than 20% of LCA studies published in the past five years reported any kind of uncertainty analysis. There are many different sources of uncertainty in LCA, which can be classified as parameter, scenario or model uncertainty. Parameter uncertainty is most often reported, although the other types are considered equally important. There are also sources of uncertainty specific to each kind of LCA—in particular related to the resolution of multi-functionality problems (i.e. allocation in attributional LCA versus the definition of market-mediated substitution scenarios in consequential LCA). However, there are currently no widely applied methods to specifically account for these sources of uncertainty other than sensitivity analysis. Monte Carlo sampling was the most popular method used for propagating uncertainty results, regardless of LCA type.
Conclusions
Data quality scores and inherent (i.e. stochastic) uncertainty data are widely available in LCI databases, and researchers should generally be able to define comparable uncertainty information for their primary data. Moreover, uncertainty propagation for parameter uncertainty is supported by LCA modelling software. There are hence no obvious barriers to quantifying parameter uncertainty in LCA studies. More standardized methods based upon context-specific data that strike the right balance between comprehensiveness and usability are, however, necessary in order to better account for both the shared and unique sources of uncertainty in attributional and consequential LCAs. More frequent and comprehensive reporting of uncertainty analysis is strongly recommended for published LCA studies. Improved practices should be encouraged and supported by peer-reviewers, editors, LCI databases and LCA software developers.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
1 Introduction
Life cycle assessment (LCA) is intended as a quantitative decision support tool. However, it is not fully accepted as such due to a perceived lack of robustness (Herrmann et al. 2014). One of the main issues impacting the reliability of LCA results is uncertainty (McKone et al. 2011), which may be very high, underestimated or, very often, simply ignored (Weidema 2009). The frequent lack of quantified uncertainties in LCA studies means that it is not possible to determine confidence levels for results, which is fundamental in most branches of empirical science. Identifying, quantifying and communicating uncertainty in LCA studies remain important areas for both methodological development and, in particular, increased implementation by practitioners (Herrmann et al. 2014).
The sources of uncertainty in LCA are numerous and have been categorized in different ways. Lloyd and Ries (2007) divide uncertainty into three main categories: model, scenario and parameter uncertainty. Here, parameter uncertainty (also called stochastic or data uncertainty) is described as uncertainty in observed or measured values arising from inherent variability in the sampled population, as well as uncertainty related to data quality. Scenario uncertainty refers to uncertainty due to normative choices made in constructing scenarios, including the choice of functional unit, time horizon, geographical scale and other methodological choices. This can also be called uncertainty due to choices. Model uncertainty comes from the structure of and the mathematical relationships defining the models themselves (including models for deriving emissions and characterization factors used in impact assessment models).
Anzeige
Many kinds of uncertainties influence LCA results. However, some types of uncertainties may be more pronounced depending on the type of study being conducted. The two main types of LCA are attributional (ALCA) and consequential (CLCA). ALCA examines a snapshot of the current or past state of affairs to determine the environmental impacts that can be attributed to the product studied, assuming a static system. CLCA considers future scenarios to determine the impacts that may occur as a consequence of a change in the use, method of production, production level, etc. of a product. Weidema (2003) defines ALCAs as ‘retrospective’ studies suitable for hot-spot identification. CLCAs are defined as ‘prospective’ studies to evaluate the consequences of future changes. These changes take into account multiple product systems, which interact dynamically via assumed market-mediated product substitution effects. These differences in modelling approach mean that different processes will be included within the system boundary, depending on whether an attributional or consequential LCA is being performed. All processes related to the production of the functional unit are included in an ALCA, and those processes affected by a change in the functional unit are included in a CLCA (Weidema 2003). Methods for solving multi-functionality problems in each type are also different—allocation for ALCA and system expansion for CLCA (Weidema 2003). ALCA and CLCA may also use different data. Ekvall et al. (2016) describe ALCA as using average data and CLCA as utilizing marginal data for modelling changes. These differences can affect the types and relative importance of uncertainties associated with each type of LCA, and how this uncertainty should be analysed. Weidema (2003) states that uncertainty in CLCA may be quite large but is able to be estimated and controlled, whereas the uncertainty in ALCA (i.e. arising from not considering market interactions) is not knowable or controllable.
In order to quantify uncertainty due to data quality in LCA, Weidema and Wesnaes (1996) proposed a semi-quantitative pedigree matrix, adapted from Functowicz and Ravetz (1990), which includes measures for data reliability, completeness, temporal correlation, geographical correlation, and further technological correlation. These indicators are intended to be sufficient to describe the data quality in terms of consistency with the study in which they are being used, and to be used in conjunction with measures of inherent uncertainty (Weidema and Wesnaes 1996). However, the pedigree matrix only addresses uncertainty due to data quality (parameter uncertainty), not model or scenario uncertainty. Moreover, it should be remembered that the pedigree matrix categories, score levels and associated standard deviations are highly generic and merit continued development and sector-specific adaptation by the LCA community in order to improve their utility for contributing to uncertainty quantification and assessment.
Uncertainty analysis in LCA involves calculating the total uncertainty of the outcome of a study based on the uncertainty of all the parameters and model choices of the modelled product system, which enables estimation of a confidence interval for the results. This is especially important since there is often a large amount of uncertainty in LCA models, such that representing the results as point values is misleading (Björklund 2002). There are many possible mathematical approaches for propagating uncertainty parameters in unit process data and characterization factors, the most common approach being Monte Carlo sampling (Groen et al. 2014). Other approaches include Taylor series expansion and fuzzy interval arithmetic. In essence, these methods all perform the same function, which is to propagate the uncertainty of the parameters. However, they have different practical efficiencies depending on the situation. Given the differences in LCA methodologies and sources of uncertainty, it is worth considering which methods of uncertainty analysis are appropriate for each type of LCA as well as the relative importance of different sources of uncertainty that are either common or unique to ALCA and CLCA.
There has been research on uncertainty analysis and the types of uncertainty in ALCA (for example, see Björklund 2002), which mostly only consider certain aspects of uncertainty, variability or unrepresentativeness. However, literature addressing uncertainty in CLCA and the differences between ALCA and CLCA with respect to uncertainty is almost non-existent. Due to the methodological differences between these two types of LCA, it is possible that the major sources of uncertainty will differ and also that different methods may be required to address them. The relative importance of these sources of uncertainty, and how they are best quantified, is presently unclear.
Anzeige
The aims of this paper are hence two-fold. First is to identify and evaluate the frequency of uncertainty reporting in LCA studies (including ALCA, CLCA and any combinations of the two). The common sources and methods of quantification of uncertainty are also reviewed - both those that are common to all LCA studies, as well as unique to either ALCA or CLCA. This includes identifying any current issues in uncertainty analysis methods and reporting. Second is to determine whether or not the methods that are commonly applied for uncertainty assessment in attributional LCA, and the kinds of uncertainty that are considered, are similarly applicable in consequential life cycle modelling contexts. It is also determined whether additional methodological development may be necessary to support improved quantification and communication of both shared and unique sources of uncertainty with respect to both ALCA and CLCA model results.
2 Methods
A broad suite of relevant LCA methods and theoretical literature that focused on uncertainty was reviewed in order to determine current recommendations, best practices and challenges. Publications discussing uncertainty in ALCA and/or CLCA were identified using Web of Science and Google Scholar and the keywords ‘uncertainty’, ‘error’, ‘variability’, ‘consequential’ and ‘life cycle assessment’. Papers were selected based on their relevance to answering research questions regarding sources, analysis or communication of uncertainty/error/variability in either ALCA, CLCA, or both.
In order to determine the current state of uncertainty analysis and reporting in LCA, a review was subsequently performed of all relevant papers in the journals publishing the greatest number of LCA studies from January 2014 to August 2018. The journals included were the International Journal of Life Cycle Assessment (IJLCA), Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling and Journal of Industrial Ecology, each of which returned at least 150 results using keywords ‘life cycle assessment’ in the Web of Science Core Collection for the years 2014–2018.
2.1 Reporting of uncertainty
The type of LCA study (attributional or consequential) was recorded, as well as whether or not uncertainty analysis was performed, or if uncertainty was even mentioned at all in the publication. The type of study was determined first based on any indications of LCA study type (ALCA or CLCA) by the authors, and second based on the research questions and methods if no LCA type was indicated. If the authors said that an uncertainty analysis was performed, the paper was categorized as ‘performed’, regardless of whether or not the results of the uncertainty analysis were presented in the paper. This included all types of uncertainty analysis (i.e. both quantitative and qualitative parameter uncertainty analysis as well as analysis of model and scenario uncertainty). However, simply a ‘sensitivity analysis’ with no mention of uncertainty was not included as uncertainty analysis since these two analyses have fundamentally different goals. In the case of IJLCA, as it is an LCA-focused journal, any discussion of uncertainty analysis was also recorded for non-LCA case studies, including review papers and methodological papers. However, for all other (non LCA-specific) journals, no analysis was performed of non-LCA studies since those studies are not relevant to the research questions. The total number of each type of paper was tabulated and graphed, along with the number of each LCA type for each category of uncertainty reporting. The rate of reporting of uncertainty (i.e. % of papers) annually over this interval was also calculated.
2.2 Sources and quantification of uncertainty
Both the LCA case studies and theoretical/methodological publications regarding uncertainty in LCA were reviewed in order to identify the methods for quantification of uncertainty in either ALCA or CLCA. The broad category sources of uncertainty (parameter, scenario and model) mentioned in the LCA case studies were tabulated, as well as any specific sources of uncertainty (within the broad categories) mentioned. Since not all studies reported specific sources of uncertainty, methods documents discussing uncertainty in LCA were reviewed in order to gain broader insight into sources of uncertainty common to all LCA studies or unique to ALCA or CLCA. Similarly, the methods of propagation and quantification of uncertainty were identified from the LCA case studies (categorized by LCA type). If any paper utilized more than one uncertainty analysis method, this was categorized as one instance of each method employed. Based on this review, current limitations and potential for improvement of the availability/suitability of methods to address uncertainty in ALCA and CLCA were evaluated.
2.3 Uncertainty analysis/propagation specific to ALCA and CLCA
A review of methods/guidance documents discussing uncertainty in LCA was also conducted to determine the current state-of-the-art with regard to best practices, general areas of agreement or disagreement, and any gaps in the literature or outstanding questions related to uncertainty propagation in both ALCA and CLCA, or common to both types. The aforementioned LCA case studies published from 2014 to 2018 were assessed against these best practices in order to evaluate the current quality of uncertainty analysis in published LCA research and to make recommendations for improvement, if necessary.
2.4 Uncertainty analysis by sector
Finally, the LCA case studies from 2014 to 2018 were sorted according to ISIC economic sector (United Nations 2008). These sectors were then mapped to the different sectors for which a standard base uncertainty is provided in Frischknecht et al. (2005). This was done to compare the size of the base uncertainty for a given sector (intended to be representative of inherent stochasticity) to the relative proportion of studies within that sector that performed uncertainty analysis.
3 Results and discussion
3.1 Reporting of uncertainty
A total of 2687 LCA case studies published in the International Journal of Life Cycle Assessment (731 studies), Journal of Cleaner Production (1426 studies), Sustainability (147 studies), Applied Energy (231 studies), Science of the Total Environment (174 studies), Resources, Conservation and Recycling (148 studies), and Journal of Industrial Ecology (140 studies) from January 2014 to August 2018 were identified for review. The overwhelming majority (94%) of these were strictly attributional LCA studies (Table 1). Only 6% were either CLCA (5%) or a combination of ALCA and CLCA (1%). Among these, 19% of ALCA studies included some type of uncertainty analysis (including quantitative and qualitative analysis of parameter uncertainty, as well as scenario/sensitivity analysis of scenario/model uncertainty) (Fig. 1). In comparison, 15% of CLCA studies reported performing uncertainty analysis, as did 22% of combined ALCA/CLCA studies. There were also 2% of ALCA studies and 1% of CLCA studies that reported that uncertainty analysis was not performed due to methodological/data limitations and rather suggested it as a future step. Overall, the majority of LCA studies (79%) did not perform any kind of uncertainty analysis, and a large fraction did not even mention uncertainty at all (43%). In publications in IJLCA that were not LCA case studies, including reviews, methodological proposals, etc., the proportion that addressed the topic of uncertainty was lower still (14%) (Table S1, Electronic Supplementary Material - ESM). Moreover, there does not appear to be an increase over time in the proportion of LCA studies that include an uncertainty analysis despite the increase in total LCA studies, in fact the percentage drops from 21 to 17% over the period studied (Fig. 2). Only aggregated data (all journals together) for uncertainty reporting are presented in the current document (Figure 1). Detailed data are available in Tables S1–S14 (ESM) for individual journals.
Table 1
Numbers of life cycle assessment studies reviewed from January 2014–August 2018 in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology, categorized by LCA type
Type of LCA
Number of studies reviewed
Attributional
2519 (94%)
Consequential
136 (5%)
Combination attributional and consequential
32 (1%)
Total
2687
Fig. 1
Types of LCA studies and reporting of uncertainty in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology from January 2014–August 2018. See ESM1 for complete list of citations. a) Attributional studies (n = 2522). b) Consequential studies (n = 136). c) Combination attributional and consequential studies (n = 32)
Fig. 2
Reported uncertainty analysis in LCA case studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology from 2014–2017. Note that 2018 was excluded because there were only data for eight months. See ESM1 for complete list of citations
×
×
3.2 Sources and quantification of uncertainty
3.2.1 Sources common to both ALCA and CLCA, their importance, and their rates of consideration
Table 2 lists specific sources of uncertainty common to all LCAs, as well as those specific to either ALCA or CLCA. These lists are based on the types of uncertainty seen in either only ALCA or CLCA studies, or in both LCA types from the review of LCA studies from 2014 to 2018. Most sources of uncertainty are common to both types. These sources include data variability and quality (parameter uncertainty), many kinds of methodological choices (scenario uncertainty) and impact assessment methods (model uncertainty).
Table 2
Sources of uncertainty in LCA in general and specific to ALCA and CLCA from LCA studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology reviewed from January 2014-August 2018
Source of Uncertainty Common to ALCA and CLCA
System boundaries; mathematical relationships in model; data variability, availability, accuracy, representativeness; time horizon of study/timing of emissions; stakeholder interpretation of results; transformative technology; human behaviour; methodological choices (functional unit, multi-functionality, impact assessment methods); normalization; weighting; lack of knowledge of biological processes (e.g. soil carbon, land change, ecosystem services); assumptions of linearity and fixed relationships
ALCA-specific Uncertainty
Co-product allocation
CLCA-specific Uncertainty
Behavioural-economic impacts: change over time (technical change or input substitution), horizontal competition for intermediate goods, impact on consumption (rebound), strategic competitive behaviour, impact of technology, future production costs, future demand; system expansion/substitution
Several studies have evaluated the relative importance of the different types of uncertainty in LCA, with contradicting results. As reviewed by Lloyd and Ries (2007), some suggested parameter uncertainty to be the most important (for example, Huijbregts et al. 2003), some concluded that uncertainty in characterization and valuation factors (or model and scenario uncertainty) are more important (Steen 1997), and others claimed that all were important (Huijbregts et al. 2005). In reality, their relative importance will likely often be context-specific—for example, depending on which LCIA methods and impact factors are considered (Henriksson 2015). That said, no LCA study identified for this review has included analysis of all sources of uncertainty in a single, quantitative uncertainty assessment, neither is it likely that this is even possible (there will always be unknown unknowns). Lloyd and Ries (2007) found that parameter uncertainty was most commonly reported and that important sources of uncertainty may often be overlooked. This is an important issue since including some, but not all, sources of uncertainty can lead to incorrect and misleading results (Uusitalo et al. 2015).
Uncertainty from the impact assessment stage of LCA can sometimes (but not always) be more subjective than inventory uncertainty, due to the uncertain nature of actual environmental impacts as well as individual preferences, which leads to more assumptions and value judgements being required (Lloyd and Ries 2007). This includes the choice of characterization factors, category indicators, etc. and their representativeness. Weidema et al. (2013) found that uncertainty data is more often reported for LCI data than LCIA, but that the latter source of uncertainty may be just as important.
Tables 3, 4, and 5 list the number of ALCA, CLCA and combination ALCA/CLCA studies that were reviewed that account for each broad category of uncertainty, including the method of uncertainty analysis. This only includes those studies which were previously identified as having performed an uncertainty analysis. Many studies accounted for more than one source of uncertainty; therefore, the data was also visualized using Venn Diagrams (Fig. 3a, b). Among ALCAs in which some form of uncertainty analysis was reported, 410 out of 470 studies reviewed (87%) accounted for parameter uncertainty, 77 (16%) for scenario uncertainty and 54 (11%) for model uncertainty (Fig. 3a). Only eight studies (less than 2%) included all three sources of uncertainty in their analysis. Ninety studies (18%) included a combination of two different sources. Twenty-seven studies (6%) did not indicate the sources of uncertainty but still indicated that they performed an uncertainty assessment. Others stated that they did not analyse any uncertainty but parameter uncertainty, citing lack of suitable methods (Romero-Gámez et al. 2017).
Table 3
Sources of uncertainty and analysis type, categorized by parameter, scenario or model uncertainty, in attributional LCA studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology reviewed from January 2014-August 2018
Parameter only
Scenario only
Model only
Parameter and scenario
Parameter and model
Scenario and model
Parameter, scenario and model
Not indicated
Monte Carlo
217
3
6
12
23
0
4
22
Fuzzy
3
0
0
1
0
0
0
0
Not indicated/other
21
2
1
3
2
0
0
3
Probabilistic (other than Monte Carlo)
11
0
1
0
2
0
1
0
Qualitative
4
0
0
1
1
0
1
0
Regression
1
0
0
0
0
0
0
0
Scenario
7
6
1
12
1
0
0
2
Sensitivity
55
9
1
17
4
3
2
0
Taylor series
4
0
0
0
0
0
0
0
Table 4
Sources of uncertainty and analysis type, categorized by parameter, scenario or model uncertainty, in consequential LCA studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology reviewed from January 2014-August 2018
Parameter only
Scenario only
Model only
Parameter and scenario
Parameter and model
Scenario and model
Parameter, scenario and model
Not indicated
Monte Carlo
7
0
0
2
3
0
0
0
Fuzzy
0
0
0
0
0
0
0
0
Not indicated/other
0
0
0
0
0
0
0
0
Probabilistic other
2
0
0
0
0
0
0
0
Qualitative
0
0
0
0
0
0
0
0
Regression
0
0
0
0
0
0
0
0
Scenario
1
0
0
1
0
0
0
0
Sensitivity
0
1
0
2
0
0
0
0
Taylor series
0
0
0
0
0
0
0
0
Table 5
Sources of uncertainty and analysis type, categorized by parameter, scenario or model uncertainty, in combination attributional and consequential LCA studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology reviewed from January 2014-August 2018
Parameter only
Scenario only
Model only
Parameter and scenario
Parameter and model
Scenario and model
Parameter, scenario and model
Not indicated
Monte Carlo
2
0
0
0
0
0
0
0
Fuzzy
0
0
0
0
0
0
0
0
Not indicated/other
1
0
0
0
0
0
0
0
Probabilistic other
0
0
0
0
0
0
0
0
Qualitative
1
0
0
0
0
0
0
0
Regression
0
0
0
0
0
0
0
0
Scenario
0
1
0
0
0
0
0
0
Sensitivity
1
0
0
0
0
0
0
0
Taylor series
0
0
0
0
0
0
0
0
Fig. 3
Sources of uncertainty included in LCA studies that performed an uncertainty analysis in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology from January 2014–August 2018. a) Attributional studies (n = 470). b) Consequential studies (n = 19)
×
In CLCAs, 18 out of 19 studies (95%) that performed uncertainty analysis included parameter uncertainty as a source of uncertainty in their analysis, six (32%) included scenario uncertainty, and three (16%) included model uncertainty (Fig. 3b). None of these studies included all three types of uncertainty analysis, and eight (42%) included two types. In combination ALCA and CLCA studies, five out of six studies (83%) listed parameter uncertainty as the only source of uncertainty analysed, and one (17%) listed scenario uncertainty. No combination ALCA/CLCA studies accounted for more than one type of uncertainty.
Common sources of model uncertainty were characterization factors and emission factors (ex. Dong and Ng 2014; Aubin et al. 2018). Sources of scenario uncertainty were from modelling choices including the definition of the functional unit, system boundary, and handling of co-products (ex. Tu et al. 2018). Parameter uncertainty was by far the most common source of uncertainty considered, from both inherent data variability and data quality uncertainty quantified using the data quality indicators of the pedigree matrix (ex. Galle et al. 2017). See Table 2 for a list of sources of uncertainty in the LCA studies reviewed.
3.2.2 Uncertainty specific to ALCA
The main source of uncertainty specific to ALCA comes from solving multi-functionality problems through allocation, which is not used in CLCA (Pelletier et al. 2015). Scenario uncertainty from the choice of allocation method was indicated as a source of uncertainty in many of the ALCA studies reviewed, but not in the CLCA studies (Table 2). For example, Van Stappen et al. (2018) found a 10–26% change in results depending on the type of allocation used (economic or mass). It should be noted that the uncertainty that comes from the choice of allocation method is separate from the inherent variability related to the specific factors used for a particular allocation method (for example, price differences over time and space when economic allocation is used). This second type of variability is additive to LCI variability, as it is a type of data uncertainty. The uncertainty due to allocation choice, like other sources of scenario or choice uncertainty, was often analysed using sensitivity analysis. Of the 77 ALCA studies that accounted for scenario uncertainty, 49 (64%) used scenario or sensitivity analysis to assess it (Table 3). However only 16% of studies included scenario uncertainty in their uncertainty analysis, whereas 87% included parameter uncertainty. See Fig. 3a and Table 2 for an overview of sources of uncertainty in ALCA.
3.2.3 Uncertainty specific to CLCA
CLCA differs from ALCA in that it considers the market-mediated changes and associated environmental impacts in the broader economic system that may arise due to a change in a given product system. However, the identification of marginal technologies and the pathways and magnitude of the assumed substitution are quite difficult in practice (Ekvall and Weidema 2004). This is the main source of uncertainty specific to CLCA (Table 2). Further uncertainty is also added with respect to the definition of the system boundaries (Mathiesen et al. 2009). Overall, the inclusion of such assumed economic effects may introduce considerable uncertainty into CLCA due to the uncertainty of both economic activity and future predictions in general (Ekvall and Weidema 2004).
Process-based CLCA is most simply modelled as a linear process which has an associated set of assumptions. It assumes fixed input/output coefficients, infinite elasticity of inputs, and an adequate market capacity for co-products (Yang and Heijungs 2018). In other words, it creates a linear extrapolation of the original system to model a change in the system. As these assumptions do not accurately reflect reality in many cases, this is a source of uncertainty that can impede CLCA as a decision support tool (Pelletier et al. 2015). The problem is not limited to CLCA but is also a well-recognized problem of econometric policy evaluation in economics (Lucas 1976). However, some CLCA methodologies include more sophisticated systems based on marginal markets (e.g. Schmidt 2008). To help avoid erroneous conclusions, Yang and Heijungs (2018) suggest using process-based models in conjunction with other types of models such as general equilibrium models, linear optimisation models, system dynamics and integrated assessment models when making decisions, especially for economically and environmentally important policies. All of these methods will have their own characteristic types and degrees of uncertainty. In particular, they require assumptions regarding the preferences and objectives governing the decision making of economic agents.
Most CLCA studies that were identified as having performed an uncertainty analysis accounted for parameter uncertainty, with only 32% of CLCAs and 17% of combination ALCA/CLCAs accounting for scenario uncertainty, which is the type of uncertainty introduced by the identification of marginal technologies (Tables 4 and 5). See Fig. 3b and Table 2 for an overview of sources of uncertainty in CLCA.
3.3 Methods for uncertainty propagation in LCA
Parameter uncertainty is the most commonly analysed source of uncertainty in LCA studies and is most often undertaken using stochastic modelling (mostly Monte Carlo (MC) simulation). MC is most often available and used in LCA software but requires a large number of samples and thus considerable computing time. Other than MC, alternative approaches for quantifying uncertainty in LCA include fuzzy data sets, analytical uncertainty propagation, probabilistic simulation, and Bayesian statistics (Björklund 2002). However, the most appropriate method of uncertainty propagation will differ depending upon the parameter data available as well as the goal and scope of the study, in terms of necessary specificity and amount of data. It will also depend on the availability and cost-effectiveness of software. Currently, OpenLCA is an example of a free and open source LCA software with uncertainty analysis capability (using MC), although only data variability, uncertainty due to data quality and uncertainty due to characterization factors can be included in the analysis. There is also open source statistics software such as R that can be used for uncertainty analysis.
According to Lloyd and Ries (2007), analytical uncertainty propagation, interval calculations, and fuzzy data sets can be less accurate than stochastic sampling. Groen and Heijungs (2017) found that both analytical and sampling approaches can yield approximately equal variance and sensitivity indices, depending on the study. Heijungs and Lenzen (2014) found that either analytical or sampling techniques may be preferable, depending on the situation, and that analytical approaches are faster but can be less reliable for large uncertainties. Groen et al. (2014) tested multiple methods of uncertainty propagation and found that sampling methods yielded more usable data than fuzzy interval or analytical uncertainty propagation. They also found that Latin Hypercube and quasi Monte Carlo sampling were more accurate and sometimes even faster than MC sampling. Di Maria et al. (2016) proposed a new probabilistic method of uncertainty propagation based on variance and geometric mean, which produced comparable results to MC. Only some of these propagation methods allow for dependent sampling (Heijungs et al. 2017), something that is desirable for only considering relative results and paired-sampled statistics (Henriksson et al. 2015). Despite the various options, MC remains the standard uncertainty propagation method most commonly available and used in LCA software.
MC simulation, or other Monte Carlo-type propagation techniques such as Latin Hypercube, were the most commonly applied methods of stochastic modelling in the LCA studies reviewed. Of the 502 LCA studies reviewed which reported uncertainty analysis, 301 (60%) employed MC (Fig. 4, Tables S15–S21 - ESM). Several studies applied MC to propagate model uncertainty from characterization factors or emission factors, or scenario uncertainty due to choices in addition to just parameter uncertainty (ex. Meinrenken and Lackner 2015; Usack et al. 2018). Some studies combined MC simulation with other uncertainty analysis methods such as semi-quantitative analysis based on expert judgement, since MC generally is commonly only used to account for data uncertainty, not uncertainty from the models or impact assessment methods (Van Stappen et al. 2018). However, it was most commonly used only for parameter uncertainty (82% of studies). Other methods of quantitative stochastic uncertainty analysis performed in the LCA studies reviewed included Taylor series (1%). Taylor series analysis has been suggested as another method to account for uncertainty from both data and other sources, and can also be used in conjunction with DQI scores to propagate individual parameter uncertainties when these input parameters do not have a known range of variability (Cordes et al. 2016). A small number of probabilistic analyses other than Monte Carlo-type (4%), fuzzy logic (1%) and regression analysis (< 1%) were also reported. Overall, of the studies that reported having performed an uncertainty analysis, 2% performed a purely qualitative analysis, 7% performed scenario analysis, and 19% performed a sensitivity analysis. An additional 7% did not report how the uncertainty analysis was performed.
3.3.1 Uncertainty propagation in ALCA studies
Mendoza Beltran et al. (2016) developed a methodology for simultaneously propagating data uncertainty and uncertainty due to methodological choices, specifically allocation method. MC simulations are used for the propagation of unit process data uncertainty, as well as the choice of allocation method. This is more representative of the overall uncertainty and can be quicker to perform than modelling each allocation method as a separate scenario, including a MC analysis of each scenario. Using a more efficient statistical propagation method such as Latin Hypercube sampling is also possible with this method. The method can also be applied to other methodological choices, not only allocation.
Among the reviewed studies, Monte Carlo is the most common method of uncertainty analysis in ALCA, accounting for 287 out of 475 (60%) ALCA studies that reported uncertainty analysis (Fig. 4a, Tables S15–S21 - ESM). Other methods of uncertainty analysis employed in the ALCA studies reviewed were Taylor series (1%), probabilistic (other than MC) (3%), fuzzy logic (1%) and regression analysis (< 1%). A small number (2%) of studies used qualitative methods. Scenario modelling accounted for 6%, and sensitivity analysis 19%. An additional 7% of studies did not report the type of uncertainty analysis used or used some other kind of uncertainty analysis.
Fig. 4
Types of uncertainty analysis performed in ALCA and CLCA studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology from January 2014-August 2018. Studies that employed a combination of uncertainty analysis methods are counted under each method. The percentages are calculated based on the sum of all the categories, including those studies that employed multiple methods and were included in more than one category. See ESM1 for complete list of citations. a) Attributional studies (n = 482). b) Consequential studies (n = 20). c) Combination attributional and consequential studies (n = 7)
×
3.3.2 Uncertainty propagation in CLCA studies
Larrea-Gallegos et al. (2019) performed an uncertainty analysis in their CLCA of pisco grape production. The marginal effects of the change in pisco production were modelled using the stochastic technology-of-choice model (STCM), in which the uncertainty of any element of the model can be assessed using its probability density function and, combined with stochastic aspects of parameters, MC simulation. Plevin et al. (2010) studied bioenergy production using CLCA. To determine the potential range of the impact of indirect land use change associated with bioenergy production, they subjectively created uncertainty distributions for each parameter and propagated the uncertainty using MC simulation, and also determined the contribution of each parameter to the overall uncertainty. This resulted in large ranges—larger than reported in previous literature on the subject - since this study combined the uncertainties of all the input parameters (Plevin et al. 2010). Much of this variance is from uncertainty in modelling choices and inherent stochasticity of parameters.
The newest version of the ecoinvent database (version 3) now includes consequential LCI models (Wernet et al. 2016). The consequential models use substitution instead of allocation and are intended to be used in CLCAs. Products are attributed all benefits or burdens associated with their resource inputs and emissions but are then credited for any co-products that substitute other products. However, there is no change in the pedigree matrix traditionally used in ecoinvent for the consequential system model, meaning that there are no additional categories to take into account unique sources of uncertainty specific to CLCA. Practitioners of consequential modelling should look to other fields, such as economics, to determine how best to quantify and analyse the unique and potentially very large sources of uncertainty in CLCA that arise from the identification of the assumed market-mediated substitutions, the affected markets and the magnitude of the assumed substitution effect.
There are also limitations when using the attributional framework for consequential modelling (Yang 2016). Real life scenarios are not likely to scale linearly due to such factors as physical constraints to expansion and market-mediated human behaviours. However, the computable general equilibrium models often used in CLCA (Earles and Halog 2011) have their own assumptions such as rational expectations which can also increase the uncertainty of the result. Yang (2016) proposed a two-step approach to conducting a CLCA using the more familiar ALCA framework. The first step is to conduct an ALCA to determine the status quo and any hot-spots. The second step is to conduct scenarios (and likely collect more data) based on the change to be studied in a consequential framework. In the end, this yields a range of scenario output values. This process requires marginal input/output coefficients to replace the average coefficients used in ALCA. However, Weidema (2017) argues that the attributional framework is not appropriate to answer consequential questions and contributes a large amount of uncertainty if used. They compared the same systems in the consequential and attributional models in ecoinvent version 3 and found that in most cases, there was at least a 10% difference in impact between the two systems, up to an order of magnitude difference. The sources of these differences came from the following five categories: marginal suppliers that are very different from the average, specialty products, multiple determining products, by-products from treatment activities and determining products heavily influenced by by-products. On this basis, Weidema (2017) suggests that using the consequential models in ecoinvent for CLCA studies removes an important source of error from CLCA.
Modelling many different scenarios can help alleviate the risk of under-representing uncertainty and presenting results as too certain (Ekvall and Weidema 2004). Ekvall and Weidema (2004) also suggest cutting off the system boundaries where the uncertainty becomes too large to support any realistic decision-making, and that short-term fluctuations in marginal technology can be ignored in favour of long-term effects. Mathiesen et al. (2009) suggest performing a sensitivity analysis to determine the importance of switching between different marginal technologies. In some cases in the literature, this can have a significant impact on the outcome of the study (Mathiesen et al. 2009). They also suggest to test the LCA results against several fundamentally different marginal technology options in order for the results to not become too sensitive to the uncertainty introduced by marginal technology changes.
Overall, 14 out of 27 (52%) CLCA (or combination ALCA/CLCA) studies published in the past five years that included uncertainty analysis used MC (Fig. 4b,c, Tables S15–S21 - ESM). Two studies (7%) used other probabilistic methods. One (4%) used qualitative analysis, four (15%) used scenario analysis, five (19%) sensitivity analysis and one (4%) did not report the method of uncertainty analysis. Roux et al. (2017) used MC analysis to propagate uncertainty based on the nine most sensitive parameters related to the marginal electricity mix. However, the majority of studies did not perform uncertainty analysis nor attempt to account for the uncertainty associated with market scenarios.
3.4 Uncertainty analysis by sector
All LCA case studies considered in the review were classified by ISIC sector and then mapped to the appropriate category according to the generic basic uncertainty values given by Frischknecht et al. (2005) (Table 6). The basic uncertainty values (geometric standard deviations) for different products are as follows: Thermal energy, electricity, semi-finished products, working material and waste treatment services 1.05; transport services 2.00; infrastructure 3.00; and primary energy carriers, metals and salts 1.05. This implies that there is the highest amount of inherent uncertainty in LCI data for the infrastructure sector, followed by transport services, thermal energy, electricity, semi-finished products, working material, waste treatment services and primary energy carriers, metals, and salts. In terms of performance of uncertainty analysis, results are different for attributional and consequential (including combination attributional and consequential) LCAs (Table 3, Tables S22–S28 - ESM). For ALCA, sector-specific studies with the highest level of reporting of uncertainty analysis focused on primary energy carriers, metals, and salts (26%), followed by infrastructure (21%), transport (21%) and finally thermal energy, electricity, semi-finished products, working material and waste treatment services studies (18%). For CLCA, the studies in the transport sector had the highest percentage of uncertainty analysis (30%), followed by thermal energy, electricity, semi-finished products, working material, waste treatment services (17%), infrastructure (11%) and primary energy carriers, metals, and salts (8%). Aggregated results are presented in Table 6, and individual journal results in Tables S22–S28 (ESM). In theory, one would anticipate that the sectors with the largest basic uncertainty should have the most studies that perform uncertainty analysis, in order to accurately quantify the large inherent uncertainty in that sector. However, this is not seen in the reporting of uncertainty analysis by sector. There are also differences between ALCA and CLCA in terms of overall percentages of reporting as well as the order of most to least reporting by sector.
Table 6
Number of studies in the International Journal of Life Cycle Assessment, Journal of Cleaner Production, Sustainability, Applied Energy, Science of the Total Environment, Resources, Conservation and Recycling, and Journal of Industrial Ecology from January 2014-August 2018 that either did or did not perform uncertainty analysis, categorized by the major industrial sectors with defined base uncertainty values (Frischknecht et al. 2005). Percentages refer to how much of the total of each sector either performed or did not perform uncertainty analysis. Within each sector, papers are further subdivided by LCA type; CLCA in this case refers to all LCA studies that employed consequential modelling, including combination ALCA/CLCA studies
Primary energy carriers, metals, salts (uncertainty factor 1.05)
ALCA
26%
74%
292
CLCA
8%
92%
48
4 Conclusions and recommendations
Despite the generally recognized importance (and widespread practice) of quantifying and communicating uncertainty associated with the results of scientific studies, uncertainty reporting within the LCA community is not yet commonplace. Less than 20% of the LCA studies published since 2014 report uncertainty analysis of any kind, and the rate of reporting does not appear to be increasing over time. In light of the potentially high levels of uncertainty in LCA studies, improved methods and practice with respect to uncertainty analysis are clearly critical to ensuring the utility of LCA as a decision-support tool. Of the many sources of uncertainty in LCA, parameter uncertainty is the main type that is accounted for with any degree of regularity among the subset of published LCA studies considered in this review that actually include uncertainty analysis. This is likely due to the lack of commonly accepted methods for quantitative analysis of model or scenario uncertainty.
There are important sources of uncertainty that are unique to either ALCA or CLCA, largely related to how multi-functionality problems are resolved (i.e. choice of allocation criterion in ALCA, and market-mediated substitutions based on identification of marginal technologies in CLCA). For the latter, uncertainty arises with respect to the identification of marginal technologies, general market trends (Mathiesen et al. 2009), assumptions regarding elasticities and the uncertainty of future predictions in general (Ekvall and Weidema 2004). However, this review of recent literature suggests that uncertainty in CLCA is currently analysed in the same manner as is prevalent for ALCA, with little apparent consideration of how to account for these unique and potentially very large sources of uncertainty.
Monte Carlo-type analysis is by far the most common method of uncertainty analysis employed in the reviewed studies, despite the potential that other methods may be as accurate and less time-consuming. The current prevalence of Monte Carlo sampling likely reflects its general applicability and the fact that it is commonly supported in commercial LCA software.
The frequency of uncertainty reporting among LCA studies for products within specific economic sectors does not align with sector group ‘base uncertainties’, as are commonly applied in the popular ecoinvent database. In general, one might expect that studies within sectors whose LCI data are characterized by the highest parameter uncertainty would have the highest level of uncertainty analysis, but this does not seem to be the case.
Based on this review of the literature, the following specific recommendations for the LCA community with respect to uncertainty assessment methods and practice have been identified:
1.
Comprehensive uncertainty analysis should be included in all published LCA studies, unless it can be justifiably excluded with regard to the goal and scope of the study. This is particularly pertinent for studies that claim ISO-14044 compliance and which advance comparative assertions. Journal editors and reviewers should begin to enforce this requirement, and should include it in their author guidelines. IJLCA, as the flagship disciplinary journal, should take the lead in ‘raising the bar’ in this regard.
2.
More attention should be paid to uncertainty analysis in studies specific to the transport and infrastructure sectors, due to their higher basic uncertainty values (if, indeed, these values are subject to and stand up to critical scrutiny).
3.
Practitioners should consider different uncertainty propagation methods in order to select the most practical and efficient method for the study, rather than using Monte Carlo sampling by default. LCA software developers should provide a range of options for uncertainty assessment in addition to Monte Carlo sampling.
4.
Concerted efforts need to be made to enable considering all relevant types of uncertainty and variability related to LCA studies, not just parameter uncertainty. This will require development of consensus methodologies for quantifying model and scenario uncertainties. This is currently not available in most major LCA software.
5.
When performing an attributional or consequential LCA, attention should be paid to the types of uncertainty specific to the type of LCA. Methods should be agreed upon for accounting for the variety of sources of uncertainty present in both ALCA and CLCA. In particular, appropriate methods must be identified/developed for quantifying the unique and potentially very large uncertainties characteristic of CLCA. Methods for sensitivity analysis in consequential modelling could be borrowed from other fields, such as economics.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
