Since the 1960s, development of the transport system has been framed by the notion of forecasting future demand. Yet the past decade or more appears to signal some significant changes to the role of travel in society which are having a material impact on how much people travel (and may travel in the future). Coupled with the potential for major technological changes and a range of climate adaptation scenarios, the future of mobility presents today’s decision making on transport strategy and investment with a broader set of uncertainties than has previously been considered. This paper examines current mainstream practice for incorporating uncertainty into decision-making, through an illustrative case study of the highly codified approaches of the Department for Transport in England. It deconstructs the issue by first focussing on different ways in which there is an opening out or acceptance of new uncertainties and how this creates a (wider) set of potential futures. It then turns to consider how this set of futures is used, or not, in decision-making, i.e. the process of closing down uncertainty to arrive at or at least inform a decision. We demonstrate that, because the range of uncertainties has broadened in scope and scale, the traditional technocratic approach of closing down decisions through sensitivity testing is at odds with the greater breadth now being called for at the opening out stage. We conclude that transport decision-making would benefit from a rebalancing of technical depth with analytical breadth. The paper outlines a plausible new approach to opening out and closing down that is starting to be applied in practice. This approach must be accompanied by an opening up of the processes by which technical advice for decisions are reached and how uncertainties are understood and negotiated.
Notes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
An uncomfortable position
The latest in a series of road traffic forecasts from the UK Government that stretches back nearly half a century carries an acknowledgement that “[w]hile uncertainty in road traffic demand has always existed, it is perhaps now more uncertain than ever” (DfT 2018a: 27). This echoes what the National Infrastructure Commission sees as “the high levels of uncertainty which surround the decisions that need to be taken over the next thirty years” to address Britain’s infrastructure needs out to 2050 (NIC 2017: 34).
This paper is motivated by a need to respond to what appears to be a new era for transport planning in which deep uncertainty must be accounted for when informing decision making and investment. As the French philosopher Voltaire remarked some three centuries ago, “uncertainty is an uncomfortable position but certainty is an absurd one”. There is a need to make sense of the uncertainty faced and consider how it can be accommodated in a way that is appropriate both in terms of analytical rigour and ease of communication, thereby supporting rather than paralysing decision making.
Advertisement
A wicked problem
Determinants of uncertainty for the transport sector are multifaceted and include the: changing demand for travel in society, its causes and likely continuation (Marsden et al. 2018; Maltha et al. 2017); impacts of climate change on infrastructure resilience and investment needs (Buurman and Babovic 2016; Dawson et al. 2016); speed and nature of any transition away from fossil fuel powered transport (Contestabile et al. 2017; Brand et al. 2019); future way in which road travel will be priced in a shift to electric vehicles (NIC 2017; Volterra Partners and Jacobs 2017); digital age maturing with a myriad of developments in information and communications technology (ICT) that in turn influence behaviours (Lyons et al. 2018a); advent, deployment and impacts of increasingly intelligent, automated and connected vehicle technology (Shaheen et al. 2018; Rohr et al. 2015); and the traditional factors of population growth, fuel price, disposable income and land-use distribution (NIC 2016; OBR 2018).
Each of these determinants of uncertainty are subject to divergence of opinion (and underlying values) amongst experts and stakeholders. They face a lack of empirical evidence and insight. This can make way for speculation. They are each complex and yet also have complex interactions with each other. In short, it can be said that handling uncertainty constitutes a wicked problem (Head 2010; Rittel and Webber 1973) which may explain the accompanying discomfort.
A challenge to the forecast-led transport planning paradigm
Uncertainty in strategic planning is not a new phenomenon. Marchau et al. find that the way “uncertainties are handled is strongly related to assumptions one has about the future” (Marchau et al. 2010: 940). Where uncertainty is seen to be limited, or the dimensions of uncertainty are perceived to be understood, then a trend-based, forecast-led approach can be deemed to be a reasonable way of representing uncertainty (Wilkinson 2009). However, it is argued that trend-based approaches cannot deal with “processes of social transformation” (Byrne 2003: 176) in which the functional relationships are subject to change and where it is no longer possible to assign probabilities to future outcomes (Walker et al. 2010).
It is this paper’s premise that deep uncertainty is being faced. This brings with it a need to reflect upon and, we believe, revise how it is handled. We suggest that recent developments point towards a transition taking place in society away from the current regime of automobility (the motor age) towards a new regime as yet to be defined (but born of the motor age and digital age colliding and merging) (Geels et al. 2012; Lyons 2015). Such a transition could take decades to unfold but would undoubtedly impact on the case for many current investment plans which are evaluated on the trade-off of benefits and costs over periods of between 30 and 60 years into the future.
Advertisement
Of the different approaches to dealing with uncertainty beyond forecasting (as depicted above), scenario planning is perhaps the best recognised. Its genesis dates back to the 1940s and it began being embraced to great effect by Shell in the 1960s and 1970s (Chermack et al. 2001). Van der Heijden (2000, p 32) describes scenario planning as starting “from the premise that if the future is uncertain there are, in fact, multiple equally plausible futures, which we call scenarios”. Whilst there have been several examples of scenario planning exercises in transport planning (e.g. OST 2006; Banister and Hickman 2013; Zmud et al. 2013; Lyons and Davidson 2016), the process typically remains curiously upstream of, and separate from, the process of strategy and scheme development and appraisal, wherein a reliance on forecasting continues to prevail. Meanwhile, it has been suggested that the orthodox approach to transport scheme assessment has become so highly codified in the forecasting tradition that it becomes institutionally very difficult to change (Hall 1993; Marsden and McDonald 2019).
In summarising this paper’s introduction so far it can be said that we are in uncomfortable territory in which the transport sector is subject to winds of uncertain change. There is a need for appropriate means to be able to chart a robust course forwards which may run counter to the orthodox wisdom and practice.
A case-study approach
The paper explores the treatment of uncertainty through an examination of the changing approach to decision-making in England. Whilst the paper draws on literature from across the planning domain, it relates this throughout to: (1) the codified decision-making processes of the UK Government as manifested in the Treasury Green Book (HM Treasury 2003); and (2) Department for Transport forecasting and appraisal documents from the past three decades. The choice of case study results from the ability to examine how these issues have evolved over time and because of the clear documented guidance regarding how and where to treat uncertainty in decision-making. The approach adopted in the UK will be of wider interest as it is recognised as one which fits with the mainstream texts and guidance which circulate the world of practice internationally (Mackie et al. 2005; OECD 2011).
There are three main sections to the paper. The first addresses how uncertainty has been recognised and treated in the initial stages of setting the context for scheme design and evaluation. This process or these processes form part of what we refer to as opening out the set of plausible futures. The second section examines how alternative futures are then interpreted in informing and taking decisions. As Rydin (2007) notes, as well as opening out what is in scope there always has to be some way of closing down towards what is meaningful, or relevant or even reasonable to consider given the scale or nature of the decision being taken. The final section discusses the need to move beyond existing highly developed decision-making approaches to overcome the tensions of opening out and closing down. To do so, we suggest requires a re-examination of what knowledge is valued and where it is valued and accounted for in the decision-support process. This calls for an opening up of the decision-making process to reflect the new environment in which decisions are being taken. The paper outlines a plausible new approach to opening out and closing down that is starting to be applied in practice.
The paper is written both for those engaged in the detail of forecasting and assessment and those who use, or debate the implications of, such approaches. Whilst we use the detail of practice in England we try to limit the depth to that necessary to understand the important generalizable messages. We set out pointers to guidance for those wanting more detail.
Opening out
In this section we address practice to date in opening out uncertainty—namely embracing the extent of uncertainty faced. We describe the approach to considering uncertainty in the current forecast-led paradigm. Road traffic forecasts in Great Britain are used as our case study. Road traffic forms over 80% of passenger kilometres travelled in Britain (DfT 2018b) and has been the subject of frequent and transparently documented forecasting exercises since the 1970s. This allows for the treatment of uncertainty over time to be followed.
Overview of the road traffic forecasting approach
Reports of official road traffic forecasts (vehicle distance travelled) have been produced by the UK Government in (at least) the following years: 1969, 1973, 1975, 1980, 1984, 1989, 1997, 2007, 2008, 2009, 2011, 2013, 2015 and 2018. This section of the paper is based upon our review of the reports from 1989 onwards to which we have access. The UK’s Department for Transport (DfT) has evolved the National Transport Model (NTM) to produce the forecasts (DfT 2018a). The NTM is described as providing: “a systematic means of comparing the national consequences of alternative national transport policies or widely-applied local transport policies, against a range of background scenarios which take into account the major factors affecting future patterns of travel” (DfT 2010). The forecasts produced have provided one of the most significant reference points for informing and justifying policy and investment decisions in transport at a national and (with further detailed modelling) a more localised level.
Each of the sets of national road traffic forecasts (NRTF) is largely common in terms of the following. Three key inputs are considered to have the principal influence on output traffic—population (i.e. the number of tripmakers), income (GDP per capita in particular—i.e. the level of financial resource at the disposal of the tripmaker) and fuel price (i.e. a proxy for the unit cost of travel by road faced by the tripmaker). A base year in the recent past is taken as an empirical reference point from which to then forecast traffic some 20–35 years forwards.
The modelling accounts for publicly confirmed policies and investments (a point of relevance returned to later) in the transport system (typically affecting capacity) that play out through the forecast period. It is recognised that the key inputs each have uncertainty of their own and ‘high’ and ‘low’ compounded variation in the inputs [e.g. high population growth, high income growth and low fuel price (DfT 2013)] has been used as a form of sensitivity test to determine the effect on future traffic forecasts. The resulting range can be referred to as a ‘forecast fan’.
Modification of the approach
Sensitivity testing up to and including the 2013 NRTF was around a ‘central’ forecast. While a forecast fan could be seen to depict uncertainty, there was an implied and indeed sometimes stated understanding that the central projection was more likely to come to fruition than forecasts towards the outer reaches of the fan. Given that the central estimates of traffic levels have in practice never been closest to the outturn levels (Goodwin 2012) it is perhaps surprising that in most NRTF exercises the use of sensitivity tests has been predicated around the idea of a single most likely scenario with other combinations seen to be “highly unlikely” or “less likely”.
The 2015 and 2018 NRTF are each significant in that they represent a departure from this. The 2015 NRTF report made no reference to a central road traffic forecast. Instead it claimed to “employ a scenario approach to attempt to capture more of the uncertainty” (DfT 2015: 5). In a departure from only considering compound variations in the population, income and fuel price inputs, new ‘scenarios’ also considered: (1) continuation of the ongoing decline in trip rates seen in recent years (rather than using a historic average); and (2) a break in the relationship between income and travel. The 2018 NRTF exercise has gone on to consider “a wider variety of uncertainty and combining multiple issues to create plausible future states of the world” (DfT 2018a: 6). This has included, in relation to technological change, an attempt for the first time to account for future effects of electric vehicle take-up. An attempt is also made to address autonomous vehicles but this is done separately to the main set of scenarios and is carefully portrayed in terms of possible (exploratory) scenarios rather than plausible scenarios. We return to the handling of technological change below.
These most recent NRTF exercises reflect an attempt, both in the reporting narratives and in the modelled scenarios, to give greater prominence to uncertainty.1 This then raises the question of whether, in turn, the forecast fans reflect this?
Comparing forecast ranges across NRTF reports
Table 1 summarises across the NRTF reports from 1989 to 2018 how what we refer to as the degree of forecast fan has changed over time. This could be considered a measure of opening out. Across the 30 years of forecasting exercises, total road traffic (vehicle distance travelled) has never been forecast to decline (although the 2018 NRTF mid-range rate of growth is the lowest it has been—less than a third of that forecast in 1989). From 1989 to 2011 the degree of forecast fan has got smaller which could imply a growing degree of certainty about the future. This is followed by a sharp increase in the degree of forecast fan in 2013 suggesting a return of greater uncertainty. Given the move (as explained above) to ‘capture more uncertainty’ in 2015 and then in turn to consider ‘a wider variety of uncertainty’ in 2018, it might have been expected that the degree of forecast fan would have continued to increase, yet it has done the reverse.
Table 1
Comparison of opening out over time (forecast range based upon the set of scenarios considered in the report concerned)
Report year
Forecast period
Forecast range (% change in total traffic)
Degree of fana (forecast range/forecast period) base 1989 = 100
1989
1988–2025
83–142
100
1997
1996–2031
36–84
86
2007
2003–2025
20–39
54
2008
2003–2025
22–40
51
2009
2003–2035
31–50
37
2011
2010–2035
34–55
28
2013
2010–2040
23–67
91
2015
2010–2040
19–55
75
2018
2015–2050
17–51
61
aExample calculation of degree of fan
1989 forecast range = 142–83 = 59; 1989 forecast period = 2025–1988 = 37
2018 forecast range = 51–17 = 34; 2018 forecast period = 2050–2015 = 35
Degree of fan for 2018 = (34/35)/(59/37) * 100 = 61
This may be explained in part by the influence of population forecasts. Across the NRTF reports, population is consistently acknowledged as a key driver of traffic growth. The 2018 report notes “how variable ONS population projections have been over the past 50 years … an indication of the uncertainty surrounding population” (DfT 2018a: 35). It may therefore appear surprising that population uncertainty has not been accounted for in sensitivity analysis in all the reports. This is only evidently the case for the 2008, 2013 and 2018 reports (see Table 2). Table 2 indicates the population forecasts used in the NRTF over time. It also compares the forecast rate of population change and shows the extent of the range of this rate for 2013 and 2018. How, then, is population treated across the three most recent road traffic forecast exercises such that it might help explain the change in opening out represented in Table 1?:
Table 2
Comparison of population forecasts over time used in road traffic forecasting
Report year
Forecast population change (%)
Rate of population changea (population change/forecast period)-base 1989 = 100
1989
4.9
100
1997
3.3
71
2007
8.5
292
2008
14.5 (+ low migration)
498
2009
21
496
2011
18
544
2013
20 (low 10; high 30)
503
2015
19
478
2018
19 (low 13; high 24)
410
aExample calculation of rate of population change
1989 population change = 4.9; 1989 forecast period = 37 (1988–2025)
2018 population change = 19; 2018 forecast period = 35 (2015–2050)
2018 rate of population change = (19/35)/(4.9/37) * 100 = 410
2013 forecasts—the scenarios at the edges of the forecast fan were ‘low population + low GDP + high oil price’ and ‘high population + high GDP + low oil price’. Population uncertainty thus played a ‘full’ part in influencing the forecast fan (contrary to the previous reports2).
2015 forecasts—no uncertainty in population was considered in the set of scenarios forming the forecast fan.
2018 forecasts—uncertainty in population is considered through two scenarios but not in combination with uncertainty in GDP and oil price as it was in the 2013 exercise. These two scenarios do not define the high and the low end of the forecast fan as they did in 2013. These instead are defined by: (1) a scenario that considers ongoing decline in trip rate along with ongoing decline in young people’s driving licence holding; and (2) a scenario that considers a strong shift to zero emission vehicles (see further below). Neither of these scenarios consider, as far as we are aware, compound effects of population uncertainty (high/low migration).
It is also important to note that the NRTF make use of population forecasts provided by the Office for National Statistics. Road traffic forecast sensitivity analysis or scenarios relating to population must be consistent with the ONS population forecasts. As such, the judgement on the treatment of population uncertainty is, in part, ‘upstream’ of the NRTF process.
From our consideration so far of the approach that is taken to opening out in UK road traffic forecasting, the following is apparent. For each NRTF exercise, how and to what extent uncertainty over input drivers of road traffic is accounted for is a matter of judgement. Judgement determines which scenarios comprise the set considered in a given forecasting exercise and in turn influence the forecast fan. The scope of the uncertainty is bounded by institutional decisions about which uncertainties are allowed into a particular forecasting exercise both individually and in combination (given the structure of the model on which the forecasts are based). A critique of the reasoned selection of input variables is not new (De Jong et al. 2007; Goodwin 2012), other than to note that the inclusion, exclusion or approach to selecting the range of variables has not been systematic.
We cannot really be sure, therefore, that the forecast fan has been widening or narrowing from one exercise to the next on the basis of variation in traditional input variables or on the basis of different assumptions about the treatment of those variables in the sensitivity analysis or selection of scenarios to include. In turn, it is not transparent as to whether any adjustments over time are due to a changing world view about the likelihood of some outturns or due to the politics or practicalities of the day (e.g. aligning with, or not prejudging, forecasts which come from elsewhere across government). What can be said is that even with the inclusion of new assumptions about varying trip rates and a recognised need to attempt to represent the implications of electric and autonomous vehicles, the degree of forecast fan is not as great in 2018 as it was back in 1989 when these uncertainties were not on the agenda.
Reaching beyond the traditional drivers of road traffic
The approach in NRTF 2015 and 2018 of accounting for trip rate uncertainty marks an acknowledgment of the importance, in embracing future uncertainty, of changes to the mobility system and the changing nature of how people travel. The introduction, for example, of a scenario with continued declining trip rates in NRTF 2015 produces an outturn forecast with 70bn fewer vehicle miles driven per year by 2040 than if the decline in trip rates is assumed to have stopped (22% lower growth). This variation is only slightly less than the difference between high GDP/low oil price and low GDP/high oil price scenarios (25% difference in growth) which would traditionally be considered. The observed trend in trip rate decline, whilst not fully understood, is attributable to a mix of non-transport factors and changing transport policies over the past two decades (Marsden et al. 2018; Chatterjee et al. 2018; Le Vine et al. 2017). It is neither unclear to us why social change of this nature would feature in just one scenario nor why it would be treated as entirely independent of other changes such as travel costs and disposable income.
The NRTF 2018 exercise, in attempting to reach further beyond traditional drivers of road traffic and embrace technological change, exposes the challenge of doing so. The UK government has made a public commitment to “end the sale of new conventional petrol and diesel cars and vans by 2040” and “for all new cars and vans to be effectively zero emission by 2040” (DfT 2018c: 7). While hybrid cars (powered by electricity and petrol or diesel) are to be exempt, this signals a strong move towards zero emission vehicles. The government has likewise made clear its commitment to the development and deployment of driverless vehicles. To have not accounted for such change in NRTF 2018 would have been conspicuous by its absence. Yet the challenge of doing so is apparent. With regard to connected autonomous vehicles (CAVs), NRTF 2018 stopped short of offering road traffic forecasts accounting for them. It instead offered some accompanying exploratory analysis in which the aim was to consider which aspects of CAVs road traffic forecasts would be sensitive to. With reference to possible scenarios (rather than the plausible scenarios of the main NRTF exercise) it was found that modelled traffic growth was particularly sensitive to vehicle occupancy levels.
Turning to electrification, all but one of the plausible scenarios (leading to the forecast fan) take current committed policy to equate to a 25% fleet penetration by 2050 in the scenarios’ modelling. The assumption is that current taxation and uptake incentives would only lead to a 25% uptake, despite a policy goal of all new vehicles being zero emission by 2040. The ‘shift to zero emission vehicles’ scenario aligned to the 2040 commitment assumes 97% of car and LGV mileage is powered by zero emission technologies by 2050. This scenario results in the highest forecast of traffic growth. Allied to this, the report notes that “[t]here are no assumed changes to government policy (e.g. tax changes on electricity)” (DfT 2018a: 42). This is, of course, a mirage. In a scenario where only 25% of vehicles are electric then the basis for paying for transport, largely through fuel duty, remains fairly robust. By contrast, a future state with 97% of mileage as electric, and therefore almost no fuel duty, represents a foundational shift in policy on paying for travel. It is hard to reconcile this with “no assumed changes to government policy”. This assumption is central to this scenario producing the highest forecast of traffic growth. It serves to highlight how judgements concerning how scenarios are selected and represented in the NTM can not only affect the degree of forecast fan but also the overall orientation of the fan upwards or downwards.
When is opening out sufficient?
We find it surprising, given the range of uncertainties which could now be considered in future forecasts, that we see a narrowing in the range of future anticipated growth trajectories. Our reflections above suggest that whilst there is a willingness to look at how to accommodate new and emerging trends in behaviour and technological innovation, this starts from a position where they have to be adopted as some kind of variant of the existing approach to thinking about growth. When incorporating electric vehicles and automation (independently), this has proven difficult. If change is disruptive to how and why we travel then this will indeed prove difficult to accommodate within the current tools and the bounds of judgement that are applied with their use. With an assumption that such accommodation is important, one approach would be to press for a continuation in, and redoubled effort concerning, the evolution of those bounds of judgement in how scenarios are selected, represented in the NTM and then reported on. Another approach would be to consider alternative or complementary futuring tools that offer new scope for assessing deep uncertainty.
However, we should not assume that other approaches to opening out will necessarily result in better decision making support. In a recent exercise, the UK Airports Commission’s consideration of future aviation demand attempted to deal with more pluralistic futures. It incorporated both variations in historic ‘drivers of demand’ but also scenarios of how the aviation market might develop to inform its analysis of runway development options (ITF, Thompson and Jamotta 2015). However, the review of the scenarios approach (ibid.: 20) concluded that the exercise was “unnecessarily complicated and the impacts difficult to untangle” and that it “resulted in a large volume of analysis that is difficult to absorb”.
It is apparent that opening out uncertainty and its communication to decision makers risks cognitive overload. The proposed solution in the exercise above was “to focus analysis on a single scenario” which was the initial trend based forecast model which was deemed to be “central” and “most likely” and against which other “less likely” positions might be assessed. In short, having developed scenarios which opened out uncertainty there was no clear process for subsequently dealing with this complexity. In the following section we ask whether a similar problem of how to deal with multiple futures might exist in the forecasting led approach.
Closing down
As already mentioned in the Introduction, Rydin (2007: 58) notes that, as well as opening out what is in scope for a decision, there always has to be some way of “closing down” towards what is meaningful, or relevant or even reasonable to consider given the scale or nature of the decision being taken. In other words, within the resources available for transport analysis, uncertainty has to be negotiated in such a way that courses of policy and investment can still be identified and appraised. We refer to closing down as the process of narrowing the plurality of futures for the purposes of informing targeted policymaking action. How closing down is done, as the Airports Commission review suggests, strongly bounds what uncertainties actually matter in decision-making and the extent to which decision making accommodates rather than conceals uncertainty. As with opening out, there is a need to be alert to the conscious and unconscious biases at play.
A disconnect between opening out and closing down
As noted above, the 2015 and 2018 NRTF exercises have abandoned the notion of a central projection or ‘most likely’ future. In our view this is a potentially highly significant step in terms of what it signals.3 While the degree of forecast fan in each case is smaller than that for the 2013 NRTF, the ranges of plausible total road traffic growth remain substantial: 19–55% from 2010 to 2040 for NRTF 2015 and 17–51% from 2015 to 2050 for NRTF 2018 (see Table 1). Such ranges should present a challenge to the requirements of overarching government appraisal which is to conduct a proportionate assessment of risks, uncertainties and inherent biases (HM Treasury 2003). However, the DfT’s transport appraisal guidance for considering policy options and transport schemes is—perhaps surprisingly—not yet directly related to the NRTF representation of plausible futures. A separate process of closing down uncertainty is applied as described below—and was still the case at the time of NRTF 2018 in spite of the change in approach to forecast scenarios in 2015.4
In addressing a specific scheme appraisal, DfT guidance requires the modelling of a core scenario that is based on central projection data from the National Trip End Model (NTEM) that forms part of the NTM. This central projection corresponds, in the case of the 2015 NRTF, to something very close to that report’s Scenario 1 (which assumes central macroeconomic estimates). Thus while not referred to as a core scenario in the 2015 set of NRTFs, it is taken as such for specific scheme appraisal.
DfT guidance on addressing forecasting and uncertainty in specific scheme appraisal (DfT 2018d) is detailed and full coverage is beyond the scope of this paper. However, in relation to handling uncertainty around the required core scenario it has the following to say. “Most models will not be able to reflect, explicitly and fully, the uncertainty of national trends such as GDP growth, fuel price trends and vehicle efficiency changes as they will be relying on the national models underlying NTEM. Therefore it is best to test the impact of this uncertainty by using high and low growth scenarios instead.” (DfT 2018d: 7, emphasis added). A procedure for establishing high and low growth assumptions is then stipulated. This is based on adding or subtracting a proportion of base year demand to the demand for the core scenario.5
Figure 1 shows the seven scenarios from the latest NRTF 2018 alongside the upper and lower sensitivity test bands from the reference scenario that would correspond to the specific scheme appraisal guidance above on using high and low growth scenarios to test the impact of uncertainty.
Fig. 1
High and low uncertainty boundaries with mid-range NRTF 2018 scenario taken as most likely
×
The fan of uncertainty implied by the procedure for defining high and low tests (indicated by the shaded area) comfortably incorporates five of the scenarios (those relating to different assumptions on economic growth and fuel price as well as high and low population migration). Moving across the period out to 2050, two scenarios (Scenarios 6 and 7) are seen to reach and move beyond the two bounds of the test range. The transport appraisal guidance states that “[t]he core scenario will form the basis for the analysis reported in the Appraisal Summary Table (AST) and, as such, should represent the best basis for decision-making given current evidence” (DfT 2018c: 5, emphasis added). As such, scenarios 6 and 7 would be seen as test extremes. They have gone from: (1) being part of a set of plausible scenarios in the opening out of uncertainty portrayed in the national forecasts; to (2) being extreme and indeed out of bounds scenarios in the closing down process for scheme appraisal. This is in spite of the NRTF 2018 report stating that “[a]ll 7 scenarios are considered to represent plausible futures and thus all scenarios should be taken into account when using the forecast results” (DfT 2018a: 84, footnote 60).
It could perhaps be argued that the range of the uncertainty test largely covers the set of demand scenarios and the major differences occur towards the end of the assessment period, which would in any case be heavily discounted in any appraisal and so less likely to impact on the value for money of decisions. However, that is to overlook the impacts of the choice of scenario around which to assess high and low uncertainty. We now turn to this.
Figure 2 shows the same seven scenarios but this time displays the fan of high and low uncertainty which exists if another scenario from the set of plausible NRTF 2018 scenarios—scenario 6 (where the recently observed decline in trip rates is extrapolated into the future)—is taken as the core scenario for scheme appraisal. Here, the lower bound demand uncertainty test would mean schemes would need to be robust to appraisal in a world with almost zero traffic growth. The upper bound on uncertainty falls beneath four of the other scenarios for between 20 and 35 years of the assessment period. The choice of which scenario is used to bound assessment of uncertainty, amongst a set of (equally) plausible options, matters hugely to what levels of demand are considered in closing down and which scenarios are included or excluded as a result of that.
Fig. 2
High and low uncertainty boundaries with extrapolated trip reduction NRTF 2018 scenario taken as most likely
×
A regressive approach to scheme appraisal?
In closing down, the logic of no one scenario expressly being more likely than another (established in opening out) is lost. There are parallels with the experience of the Airports Commission. Closing down appears to be done in ways which reduce cognitive overload from the presentation of too many future scenarios. It seems difficult to move away from a cognitive bias for establishing some form of reference scenario which is deemed “most likely” in order to make the assessment and presentation of such an assessment manageable. Our work suggests that the assumptions allowed for, or expected, in closing down within the forecasting paradigm can very significantly influence the bounds within which the assessment by decision makers is focused.
In a scenarios-based approach, the plausibility of different future relationships between how we participate in society and how much related travel is involved can emerge. In a forecasting-based approach those changes to society are more confined—to those which the modelling tool (which was not built with such a need in mind) is able to include. In picking a single scenario within that we further close down to one set of assumptions about how past behaviour will project forward, thus undermining the principles of a scenarios approach (van Dorsser et al. 2018).
It seems the position remains uncomfortable if not indeed unsatisfactory. While efforts are being made within the forecasting paradigm to improve opening out, limitations remain; and the inconsistency between opening out and in turn closing down needs addressing to avoid the risk of closing down amounting to shutting down (concealing some of the acknowledged uncertainty revealed by opening up). In the last section of the paper we therefore move to consider recent insights that have been elicited from engagement with other transport professionals and in particular those giving their attention to the handling of uncertainty.
Opening up
Time for change
In June 2018 the UK Department for Transport published a new Appraisal and Modelling Strategy (DfT 2018e) for consultation. Recognising changes in travel behaviour combined with technological developments as key sources of uncertainty, the Strategy points to a need for “better tools to capture and communicate uncertainty to decision makers, including the development and use of scenarios” (ibid: 8). A number of means of accounting for uncertainty are identified, including: sensitivity testing; switching values (how much an input value to forecasting would need to change before the value for money category of a scheme in the appraisal process would be affected); Monte Carlo analysis (that entails its own assumptions to be aware of); and scenario testing (whereby a scheme’s performance in different plausible future contexts can be considered).
This section of the paper summarises our thoughts on next steps for practice and research in tackling these challenges. We draw on a roundtable held in Summer 2018, organised as part of the process of developing this paper, involving a number of key individuals engaged in strategic planning, appraisal and the handling of uncertainty in UK practice (for full details see Lyons 2018).
Policy challenges and professional appetite
The issues this paper has sought to tackle concern the very real set of questions facing decision-makers today, regarding how best to chart a pathway through major societal and technological change whilst achieving challenging climate change goals and more inclusive economic growth. This means finding ways to understand the new options and potential outcomes in ways which decision-makers can process and which stand up to scrutiny (in public inquiries, in legislative expenditure committees and in the eyes of electorates).
Are there plausible futures that could represent transformative change for society and transport? If so, does this not demand that we reflect accordingly upon the efficacy of our approach to informing policy and investment? Is there professional appetite to change approach?
In a recent national initiative by the Chartered Institution of Highways & Transportation (Lyons 2016), over 200 UK transport professionals engaged in a series of day-long workshops in which their views on uncertainty and on how it is handled were explored and discussed. Participants were presented with four plausible scenarios from a scenario planning exercise considering uncertain future demand for car travel. There was greater collective belief in the plausibility of a future scenario showing a substantial decline in total car traffic (vehicle distance travelled) than in the scenario depicting growth in car traffic similar to that falling comfortably within the forecast fans from NRTF 2015 and 2018. Such professional opinion is incongruous with the results of the current forecasting paradigm even accounting for its latest efforts to evolve. These professionals also collectively signalled support for moving towards a policy making approach in which uncertainty is more strongly embraced than through further development of sensitivity approaches around a preferred or reference scenario.6
A plausible new approach
If the argument to move towards accounting for more ‘pluralistic’ futures as a basis for informing decision making is accepted then there is a substantial challenge regarding how to avoid the ‘out of bounds’ dismissal of plausible scenarios (illustrated by the NRTF case study above) and the overload problem for decision makers of taking greater account of uncertainty (highlighted by the case of the Airports Commission).
Prompted by Lyons and Davidson (2016), an approach has been progressed for addressing this challenge—one that may be especially well suited to the early strategic planning and optioneering stages of the policymaking and appraisal process. It has recently been developed and applied by Transport Scotland as part of the process of revising its National Transport Strategy (Lyons et al. 2018b).7
Set against a wish to fulfil a high level vision, the aim is to consider the implications of different courses of policy action in the face of multiple plausible futures. Does a course of policy action align with the vision in terms of its influence in each plausible future considered or does it align well in some and not in others? The intention is to reconcile risk and yield in the decision making process. As such, the approach could be considered as a modified form of ‘Robust Decision-Making’ (Lempert et al. 2006; Weaver et al. 2012).8 The best course of policy action for one assumed future may give high yield in relation to the vision but may also carry a high risk of misalignment or even failure in other plausible futures. Meanwhile, a course of action which has reasonable alignment across multiple futures in terms of pursuit of the vision may offer a lower (but acceptable) yield but with lower risk.
Figure 3 illustrates the Transport Scotland approach referred to above and in a manner that seeks to address communication of uncertainty into the policymaking process without causing cognitive overload.
Fig. 3
Opening out and closing down (based on Lyons et al. 2018b)
×
Plausible future scenarios in Fig. 3 reflect different consequences from a set of input drivers according to how those drivers are assumed to play out in future. This is equivalent to the approach in NRTF 2018 of having a set of plausible scenarios. These could be viewed in appraisal terms as multiple reference ‘do-nothing’ or ‘do-minimum’ scenarios before candidate new policy options are introduced (as distinct from orthodox practice in the forecasting paradigm where only a single reference scenario is commonly considered). A candidate new policy (do-something) can then be tested by modelling its influence, in each scenario, on the set of input drivers and in turn on the outcome consequences. A Red/Amber/Green assessment is made of how well aligned a policy and its outcome is with the desired future state (vision).
In the Fig. 3 illustration, policy option A shows good alignment with addressing the vision in the face of uncertainty—an indication of reasonable yield and low risk. Meanwhile policy options B and C suggest poor alignment and yield prospects and high risk. Policy option D indicates good alignment and potentially reasonable yield but with a signal of risk of poor yield or policy failure in relation to scenario 7. Such a matrix is the product of a relatively simple (and therefore agile) modelling tool allied to, and enabling of, dialogue amongst the actors in the policymaking process. Recognition of the tool’s function as one of enabling and informing the handling of uncertainty rather than resolving the uncertainty is foundational to the approach.
Proportionality is an important analytical consideration, with value to be found in a breadth of examination of policy options within the uncertainty space rather than being drawn prematurely to a narrower in-depth examination of (perhaps fewer) options in the context of fewer do-nothing scenarios (or even only one scenario).
Changing norms
Can this approach be made to work as part of the decision making process? The answer may well rest upon whether or not it is possible to challenge and change the norms of what constitutes acceptable or proportionate analytical robustness and being able to communicate analysis clearly. Particularly at the earlier stages in the policymaking process, simpler (though not to infer less robust) analytical tools could be more effective in exploring the uncertainty space of plausible futures and enabling dialogue and development of views of actors in the process. Existing, more resource intensive, modelling tools may be better deployed to look at specific scenarios in more detail further down the line. There is also a need to recognise the place of both ‘narrative’ and ‘numbers’ in order to ensure effective engagement with actors and to communicate the credibility of, and insights from, scenarios analysis.
This approach (or approaches similar to it) involves a significant role for the human actors in the policymaking process engaging with deep uncertainty. As we have noted earlier, handling uncertainty is a wicked problem. In the face of a paucity of evidence concerning a number of interacting and complex territories of societal, technological and behavioural change, there is also likely to be a divergence of views amongst actors themselves.
One of the challenges here is in judging whether the true extent of uncertainty is being accounted for, distinguishing between probable (likely to happen), plausible (could happen) and possible (might happen) futures. Views are subjective (and further coloured by opinion on preferable (desirable) futures). Making use of the analytical tools at their disposal, it is nevertheless critical that actors in the process open-mindedly and collaboratively consider plausible and potentially very different social futures in a way that reflects well the uncertainty space (avoiding prematurely closing down the columns in Fig. 3). This requires a different approach to closing down decision-making (see van Dorsser et al. 2018). Likewise an open-mindedness is called for in considering policy options (avoiding prematurely closing down the rows in Fig. 3). Used appropriately there is the prospect of a new or revised transport planning paradigm emerging.
Confronting inertia
Well-established approaches, procedures and norms can conspire against developing and adopting new approaches that may be better able to handle uncertainty but which are unfamiliar and potentially challenging to communicate. Acknowledging uncertainty can have connotations of poor confidence and conviction in decisions being made—for example in the context of public inquiries. Professional opinion in the UK appears divided, for example, regarding whether or not ongoing use of a central projection and core scenario in scheme appraisal (as currently advised by appraisal guidance) is appropriate. The arguments against are reflected in this paper. An argument in favour is that it offers a common point of reference for scheme appraisal—both between schemes and in terms of those supporting or opposing a given scheme.
While there may be recognition of the need to evolve how uncertainty is handled, there is inertia within the forecasting paradigm to be overcome (Marsden and McDonald 2019). Inertia is compounded by depleted (public sector) resources and capabilities to address orthodox approaches let alone to evolve such approaches or introduce new ones (NAO 2018). Inertia must be addressed if we are to see approaches such as that being taken by Transport Scotland becoming normalised rather than remaining the exceptions.
One means of starting to address this would be to consider how available resources can be redistributed across the transport planning and decision making process alongside seeking to reconsider the makeup of experts required to handle uncertainty and communicate it to decision makers. There is also a need for strong leadership within professional practice if individuals are to be given the mandate, agency and support to effect change. The application of new approaches should be strongly encouraged, with a ‘learning by doing’ philosophy where experiences of those new approaches and the lessons learned are shared with others.
Research challenges
If we are to see a paradigm shift in transport planning as regards forward planning and investment in the face of uncertainty, the issues such as those above need addressing. Research will have an important part to play as it has done in the evolution of the forecasting paradigm to date. There is not scope in this paper to fully explore and formulate a research agenda. However, we would briefly point to four further issues:
Understanding a changing world—Whilst we identify above that there is considerable professional uncertainty about likely directions of future travel demand shifts, this is also a research gap (Marsden et al. 2018; Shaheen et al. 2018). The plausibility of different combinations of social change, transport technology change and behavioural adaptation (to both) is not well-understood. A deeper understanding of the extent to which different futures could unfold, given existing land-uses, cohort effects, technology transition periods and evidence of pace of change to date should be developed.
Understanding the decision makers’ perspective—It is well established that decision-makers reduce the complexity of their tasks by focussing on significant cues (Shanteau 1992). However, we find little evidence on what those cues are or how they relate to the cognitive limitations of decision-makers, particularly in the transportation assessment context (see Nellthorp and Mackie 2000 as an exception). Even were such limits to be established or understood we see no reason why step-wise approaches to decision-support could not be developed which reduced the risk in cognitive overload (see Dodgson et al. 2009). Rather than presuming what decision-makers can cope with and working back from there to the current forecasting and assessment approach, this should be empirically and iteratively tested.
Appropriate scrutiny of different categories of intervention—How uncertainty is handled and the level of scrutiny applied may vary according to different stages in an appraisal process but also according to different categories of project or policy. This concerns proportionality, particularly in the face of limited analytical resource. It would be helpful to empirically examine how different categories of schemes and interventions might become distinguishable and in turn guide which approach(es) would offer proportionate scrutiny in each category. This might differ by place (e.g. core urban areas versus inter-urban corridors) as well as by intervention (e.g. train electrification of existing lines versus constructing new lines).
Adaptive planning—Uncertainty is not a steady-state phenomenon. While the alternative approach set out in this paper can be better suited to handling deep uncertainty than the forecast-led approach, it is at risk of being static in its application. In other words, as the world continues to evolve (in some ways potentially profoundly and rapidly) the knowledge we have about trends and views about the nature of future uncertainty will also change. A candidate policy or investment may presently be judged to strike a good balance between risk and yield; yet this may change. As such, there is an important role for ongoing monitoring and review of policies and investments. It is not clear the extent to which current or evolving practice is accounting for this or has the capacity to do so. This merits further attention (see Roelich and Giesekam 2019 and Marchau et al. 2019).
Final reflections
The paper draws in detail upon direct practitioner experience and insight from the UK and examines current and evolving practice. Is has demonstrated how transport authorities and individual professionals are alive to the need to respond to deep uncertainty and are taking steps to do so. The matters involved are non-trivial and it can be said that a mode of ‘learning by doing’ is called for and this is being pursued, at least in the UK. Practitioners in other parts of the world, while implementing country-specific appraisal approaches, are likely to be familiar with the forecast-led approach to transport planning. The intention of this paper is to offer transferable insights from the specific UK approaches considered with a view to helping constructively challenge international approaches to transport planning in the face of a changing world and thereby help ensure that the discipline evolves appropriately.
This article is written at a time when trust in experts has been declining. Butler (2017) contends that this is as a result of rising expectations that, in a technological age, the answers of experts will become ‘better’ as they have more analytical capability at their disposal. However, as we have set out, improvements in the sophistication of analytical techniques are not necessarily the same as developing understanding, techniques and processes that enhance insight into what sorts of futures might unfold or what the options for decision-makers really mean.
Butler argues that there is a need for experts to be more reflective in their practices and to “avoid the temptation of relying on the method of science to be the only harbinger of truth and knowledge” (Butler 2017: 8). Greater transparency and accountability are suggested to be important in addressing what RAND refers to as ‘truth decay’ (Kavanagh and Rich 2018).
Our aim, through this article, has been to demonstrate the limits of the existing approach to forecast-led planning. Particularly for longer-term strategic planning and major infrastructure investment programmes, there is a need to find: (1) new ways to acknowledge and account for uncertainty; and (2) ways to communicate this effectively to decision-makers and the general public. Rather than being an admission that experts should not be trusted, we suggest it requires a rethinking of what attributes represent expertise and where those are best applied in decision-making processes. Greater understanding of how policy futures might unfold and of how vulnerable policy options might be to different uncertainties would simply be a shift in valuing different attributes of professional knowledge. Given the quite substantial societal change which is implied by some of the futures currently being envisaged, we suggest that this is a skill set which requires rapid development.
Acknowledgements
We are grateful to Professors Peter Mackie and John Parkin for feedback on an earlier version of this paper and to the willingness of staff at the Department for Transport to engage on the topic being addressed. We would finally like to thank the three anonymous reviewers whose feedback has helped to further improve the published version. The time for Professor Lyons was funded as part of his Mott MacDonald-sponsored Chair in Future Mobility at the University of the West of England. The time for Professor Marsden was funded through the EPSRC Dynamics of End Use Energy Demand Centre grant (EP/K011723/1).
Compliance with ethical standards
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Indeed it is perhaps worth noting that the average incidence per report page of reference to ‘uncertain’ is twice as high in NRTF 2018 as it is in NRTF 2015.
Going further back to the 1997 and 1989 exercises that did not consider population uncertainty yet which each have a degree of forecast fan comparable to that of 2013, we infer that uncertainties in fuel price and GDP were playing a bigger part than has been the case more recently.
This said, scenario 1 in NRTF 2018—which assumes central projections for population, GDP and fuel price—is referred to as the ‘reference scenario’ and is the middle scenario in terms of levels of forecast traffic growth across the seven scenarios considered.
The amount of adjustment changes over the forecast period with the square root of the number of years out from the base year multiplied by a factor p, which for roads is 2.5 per cent. We note here that the advice is for scheme appraisal. For illustrative purposes we apply the approach to the national forecasts to demonstrate what the simple assumptions at a scheme level equate to at a national scale.
It is now also embodied in a ‘six-stage vision-led approach to strategic planning for an uncertain world’ developed by the University of the West of England and Mott MacDonald called FUTURES—see www.mottmac.com/futures.
It should be noted that the Robust Decision Making approach put forward by Lempert and colleagues is quantitatively-focused and involves modelling a large number (potentially many thousands) of scenarios. The approach discussed in this paper is distinctly different. It favours a smaller number of scenarios that can be addressed both quantitatively and qualitatively (in narrative form) so as to be able to inform and facilitate dialogue amongst stakeholders regarding the robustness of different options being considered as part of a strategy.
