Spatial and temporal scales are very important when “attempting to put sustainability into practice, or in gauging the level of sustainability” (Ko, 2005).

The diverse spatial scales have different specific factors that influence the risk management decision process. According to Graymore et al. (2010), the current sustainability assessment methods used at the global, national and state scales are not entirely effective at fulfilling their goal, according to these spatial scales. The indicators are defined on the chosen spatial scale, and they capture only synoptic aspects for the scale on which they are applied.

As territory is a hierarchical structuring system with different spatial scales (neighbourhood, municipality, county, region, nation) delineated by their administrative boundaries so that risks of natural origin are nested at those various levels, it is crucial to specify on which scale sustainability of the management is to be evaluated (Fekete et al., 2010; Kienberger et al., 2013). According to their specific characteristics, each scale (micro-, meso-, and macro-levels) has to be treated separately: a variable with specific strength at one scale could seem inappropriate at another. For instance damage estimation mainly relies on assets typology at micro level and land use at meso-level. An explicit description of the spatial scale in the conceptualisation of the methodology helps to identify accurate sustainability indicators/parameters, to determine how indicators/parameters on different levels can benefit from each other, and to detect which constraints of data collection have to be faced. In general, the preciseness of analysis increases at small scale and is more generalised towards the more aggregated scale.

It is observed that small-sized hazardous events are more frequent, as consequences it is at the fine spatial scale where theorising a methodology becomes useful for risk-managers. Hence, this paper uses municipalities, which are a smaller urban spatial unit, as a meaningful or suitable scale that could lead to a more accurate framework and assist the further adaptation and application to the other spatial scales. The focus is on typical French communities with less than 2000 people (INSEE)

According to the INSEE (Institut National de la Statistique et des Etudes Economiques), the average population of the most of French communities is about 1750 inhabitants on 1 January 2008.

According to Fekete et al. (2010), in comparison to the upper scales the main pros of this scale are more detailed information, a better capture of complexity of phenomena, the use of participatory methods for data collection, a higher availability of data related to one item, a lower level of uncertainty, etc. Contrariwise, it is limited by loss of information while up-scaling the assessment.

Concerning the temporal orientation, this framework obeys a prospective logic because it is designed for ex ante assessments of decisions; it might be used to examine the sustainability of existing management measures. Because the consequences and impacts of decisions can vary over time, their sustainability must be assessed on different temporal scales. Some effects can occur immediately after implementing the decision or only after a longer or shorter interval. First, this paper argues that the temporal scale should involve the entire life or mission span of the risk management decisions. Second, planning those decisions within the context of sustainability entails planning beyond 50 or 100 years (Saunders, 2010a), requiring plans for future generations. Indeed, the decisions “should not only be taking short- and medium-term into account but also the (very) long-term, thus leading to more sustainable risk decisions” (Genserik, 2012).

Building on the evidence that accounting for different temporal perspectives can improve the level of sustainability, this framework is intended to address the assessment as a continuum (ranging from the short to the long term) when predicting the variability of the sustainability over time. This tool should facilitate strategic planning within sustainable risk management in the long term while considering the dynamic behaviour of the factors that affect the sustainability over time, such as the expected territorial dynamic; this factor helps determine the future of the territory and establish risk management requirements. In the case study, only one timescale assessment was undertaken. However, several timescale assessments should be performed in practice to appraise the evolution of the sustainable strategies over time and to determine the most sustainable decision over time, thereby generating relevant sustainable decisions for the future.

The main objective is to elaborate an indicator-based grid. Criteria and indicators can be selected using a top-down or a bottom-up approach (Franco and Montibeller, 2009; Weiland et al., 2011). Both approaches involve decomposing a complex decision into a hierarchical structure that represents the sustainability performance and is built from the input variables situated at the bottom level of the pyramid.

The top-down method is used to break down the sustainability concepts into dimensions, criteria and indicators. This deductive approach facilitates the following: the theoretical description of the objectives and the rigorous collection of the corresponding criteria and indicators from the literature. This approach should ensure that the correct conceptual information is retained with the common criteria and indicators of generic sustainability.

The bottom-up approach is used to select criteria and indicators that match the contextual requirements from the specific field whose sustainability will be assessed. This inductive approach is often rooted in the concerned field and aims to ensure that specific information related to the field at hand is accounted for. It starts with the identification of the potential consequences and effects of the field. Once the set of consequences and effects is defined, they can be grouped into indicators based on similarities. Indicators can then be grouped into criteria. Finally, the criteria can be grouped into objectives. The potential consequences and effects serve as benchmarks for the validation of which indicators and criteria are suitable to fulfil the sustainability concerns of the targeted field.

In this paper, a hybrid approach has been applied, complementarily combining bottom-up and top-down approaches. The process used to construct the assessment grid involves three steps. The first step involves identifying the criteria based on the objectives of sustainable risk management. The top-down approach was applied to identify the appropriate criteria using a combination of the principles for sustainability assessment (Gibson et al., 2005; Gibson, 2006) and those for achieving sustainable risk management Mileti (1999). The following five criteria were proposed:

Technical and functional effectiveness addresses the capability of measures to fulfil the primary function of risk management, providing protection against natural risks and reduce losses from those risks.

The bottom-up approach was exhaustively applied when inventorying the potential (direct and indirect, as well as tangible and intangible) effects and consequences (pros and cons) of risk management decisions over periods that are much longer than the lifetimes of the investments. For the natural risk management policies, the potential consequences could include a decrease/increase in casualties and disabilities (direct), a decrease/rise in economic activity (indirect), continuing damage to assets due to the residual

The residual risk is the irreducible portion of the risk associated with the potential implementation of management solutions.

These potential effects have shown which considerations are important while assessing risk management decisions; these effects were explored to identify the relevant indicators. Once these effects and consequences were identified, a literature review was completed and a set of potential indicators was created according to their relevance regarding the studied field and based on an overview of existing national and international sustainability assessment methodologies (Singh et al., 2012) and tools.

No specific indicators exist for evaluating the risk management that are universally or widely accepted (Carreño et al., 2007). Therefore, indicators were selected from various tools used to gauge sustainability in various fields. These tools are Sustainability Reporting Guidelines G4 provided by the Global Reporting Initiative, RST02 grid (France), “Boussole 21” grid (Belgium), International Urban Sustainability Indicators List (IUSIL), the reference framework for European sustainable cities, Sustainable Transportation Performance Indicators (STPI), and risk management performance criteria proposed through the action framework led by the International Strategy for Disaster Reduction (ISDR). Amongst those tools, only two refer clearly to natural risks, without making special indication on the type of natural hazards. The IUSIL considers “natural hazards” as an indicator of social criteria when assessing urban sustainability, and the ISDR framework provided a set of goals and criteria for guiding, and monitoring disaster risk reduction. These two tools could be adapted to any natural hazard by defining specifically relevant indicators or parameters.

Finally, using a collaborative and multi-disciplinary process, researchers involved in the INCERDD project shared their knowledge, experience and judgements to validate the set of the proposed indicators with regard to their relevance, applicability and other characteristics, such as measurability and accessibility to those without specific knowledge. Indeed, as the main target of indicators is to reduce the complexity of information needed by decision-makers, they should be “measurable, scientifically valid and capable of providing information for management decision-making” (Donnelly et al., 2007). Sustainability indicators have to fulfil the following requirements. They have to be: (1) relevant by showing what is essential to be known, (2) easy to understand by every actor of risk management even if he is not an expert, (3) reliable so that the information they provided will be trusted, and (4) based on accessible data so that the information will be available when it is needed. However, indicators “can be developed independently of available datasets” (Donnelly et al., 2006) and doing that could help drive production of needed data.

The experts, when selecting the components of the proposed grid, checked all the retained indicators in the light of those requirements. Indicators were retained when they meet the majority of criteria (that is to say three), which inevitably include the criterion “relevant” expressing the importance of the indicator in relation to sustainable risk management. Although it was difficult for every indicator to conform to all of these requirements, it was important that they complied as much as possible. For instance, some effects and aspects seem to be significant but remain difficult to assess, particularly regarding the social and institutional dimensions (Lekuthai and Vongvisessomjai, 2001 cited by Poulard et al., 2010). Because some of the indicators are related to intangible concerns (e.g. recreational value, quality of life), the analysis is very complex; these indicators are often assessed based on subjective assumptions. Subsequently, their estimation causes several problems, such as finding consensus on the parameters and how to measure these factors concretely.

The obtained sustainability assessment grid is constructed using a hierarchical structure that includes seventeen context-specific indicators. This grid is schematically depicted in Fig. 2 and assumes that qualitative indicators

The indicators are not necessarily split into parameters; some parameters might be split into sub-parameters.

Parameters can be identified and selected on a case-by-case basis by relying on the distinctive characteristics of the territory and its prevailing sustainable development targets. Although the objective of this paper is to provide a more precise and defined framework to assess the sustainability of risk management initiatives, the parameters were informally selected. This choice might render the framework more flexible by allowing to users to propose better parameters depending on their particular context, the specific purpose of sustainable development in their urban setting, and data availability.

Even though none of the proposed criteria and indicators is absolute, they were chosen from different domains according to key considerations briefly depicted as follows. The criterion “technical and functional effectiveness” aims to help measure a decision success in achieving damage limitation. To attain this objective, it seems appropriate to consider as indicators, hazard characteristics (intensity/magnitude, spatial extend, frequency, speed of onset, etc.), structural or physical vulnerability (typology, value and sensitivity of exposed assets), and the potential of the decision to create or exacerbate existing or new risks (both hazard and vulnerability measuring variables). The criterion “economic sustainability” intends to estimate decisions impacts on the economic productivity of the territory. The success of a strategy in enabling continued economic growth while reducing natural risks damages could be checked with indicators such as its costs, its potential avoided damages, and its ability to create or not economic value for society.

The objective of the criterion “environmental sustainability” is to appraise the capability to preserve and maintain ecological heritage. Related general targets are, at one hand, to reduce environmental vulnerability to risk, and at the other, to avoid strategies that would induce significant adverse effects on ecological heritage. Thus, the retained indicators are: “impact on the environmental vulnerability” and “environmental impacts”. To select the parameters on which the indicator “impact on the environmental vulnerability” can be split to, the stakes that, when situated in hazard prone zones, could contribute to the environmental sustainability of a territory were identified. Possible features to consider are: areas of protected natural habitats, sites with pollution potential, drinking water sources, wastewater treatment plants, volume of wastes probably resulting from risks that need to be disposed of, etc.

The criterion “social sustainability” could be split into various indicators expressing general targets related to social conditions of people. For the general target of enhancing “quality of life”, some measurable features or parameters could be: average travel time/distance to work/amenities (to show transportation trends induced by the decision), number of amenities such as shops, health care centres, recreational public spaces, etc. (because urban amenities facilitate social contact), etc. The indicators and parameters associated with “institutional sustainability” help check whether the management strategy is at the expense of another community sustainability, and if the decision-making process will actively involve stakeholders.

The multidisciplinary approach developed through INCERDD project help ensure less bias in the decision-making process by encompassing all dimensions of sustainability with a broad spectrum of influential variables. These suggested indicators should be a reference for public institutions and the private sector when making sustainable natural risk management decisions. The suggested indicators are not exclusive and should be treated as an indicative checklist of which issues to consider at a minimum when assessing possible solutions with a focus on sustainability. However, one of the challenges remains rendering the grid operational. To address this issue, the following subsection focuses on formulating a methodology to assess the sustainability performance using the grid.

Once the indicators were selected, they needed to be quantified or qualified depending on their quantitative and qualitative nature. Sustainability indicators of natural risk management can be either quantitative (e.g. number of residential buildings within the hazard prone area) or qualitative (e.g. level of recovering from disruptions). The latter may be more suitable for intangible aspects like social, and cultural concerns. For instance, “social acceptability” could be considered as a qualitative indicator, which depends on considerations such as preferences of people with regard to risk management issues, perceived fairness of decisions impacts, stakeholders' willingness to support constraints (in terms of financial costs, consistency with social customs, etc.) from decisions. Both of the two types needed to be calculated differently. Quantitative indicators provide information using numbers: they can be evaluated directly from the available data related to measured amounts or from modelling. Qualitative indicators provide information as a description using words: they could be evaluated based on a comparison to a system of references, description, perception or judgement regarding their relative importance when accurate data are limited. The obtained descriptions can then be numerically scaled using different scaling techniques (e.g. codes, matrices).

As asserted by González et al. (2013), “successful decision support tools provide information in a concise relevant format in order to inform decision-making processes”. To fulfil this objective, a sequenced, understandable and easy-to-use methodology should be used to evaluate different strategies during natural risk management. For a better understanding of each step of the methodology, the paper presents a pedagogical example to illustrate calculations regarding each equation. This example serves also as case study to demonstrate how the whole methodology may be used as decision-support tool for selecting the most sustainable risk management decision.

Assuming that the input data

The data sources (e.g. historical records, instrumental records, maps) and various methods/tools (e.g. physical and numerical models, existing mono-criterion sustainability assessment tools, expert judgements) might be used to generate input data. This framework does not indicate which methods/tools to use; the primary purpose is to generate the required input data.

While operating the methodology, users could make data screening tests (removing outliers, identifying erroneous data values, detecting missing data, etc.). Indeed, the problem of input data availability is crucial because in reality there will inevitably be some missing data. It is necessary to supplement missing data, if it is possible. An option is simply to exclude the parameters that are suffering missing data from the set of parameters for all the assessed alternatives. Failing that, users could solve this problem through several missing data imputation procedures. They could build plausible data according to similarities of the study case with other cases (external sources) or based on imputation methods such as mean – median – mode substitution, nearest neighbour interpolation, various regression techniques, etc. (OECD, 2008; Glasson-Cicognani and Berchtold, 2010). Although imputation can help minimise bias, it should always be kept in mind the incompleteness of data because imputed data are not real data. Given this background, the suggested sustainability assessment methodology is organised as follows.

The chosen reference situation should be the baseline policy. Therefore, four management strategies have been retained with the mayor's staff: the “do-nothing” or status quo strategy and three alternatives. Maintaining the status quo (S1) is to continue flood risk management through existing defences (with their inspection/maintenance works continued), assuming that no new measures are undertaken. After floods, assets are cleaned-up, repaired, restored, or replaced on the basis of financing from insurance schemes. The first alternative (S2) consists of willingly respecting regulatory constraints for new buildings. Any new construction situated within hazard prone area should be raised above the level of the QRef by piling foundations or crawl spaces. The second one (S3) consists of constructing a collective defence infrastructure. The main idea is to reinforce, and to raise the existing railway embankment along the Moselle so that it can be used as a dyke sufficiently high to hold back floods up to 30 cm higher than the QRef. The third alternative (S4) consists of willingly respecting regulatory constraints for all existing buildings situated in hazard prone areas through individual protective equipment (cofferdams, check valves, suitable seals, etc.) that prevents, limits, or delays the entrance of water into buildings.

In addition to the specific measures of flood management, each of these alternatives is associated with new development projects in flood prone areas. FloodedCity plans:

Housing and commercial infrastructure construction, mainly in the inner of a low to moderate hazard prone area of the city with a high urban development potential. Around one hundred houses are to be constructed stepwise between 2017 and 2025, elevating them on pilings above the level of the most severe flood (QRef), and keeping in mind that any loss of flood storage due to embankment must be compensated for by the reduction in level of nearby ground (such that the same volume is available at every flood level before and after the project).

Applying the suggested methodology to flood risk management involves using appropriate parameters sets (quantitative indicators), particularly regarding the criterion “technical and functional effectiveness”. A review of literature on flood risk management helps identify relevant parameters that best fit FloodedCity local conditions, as well as flood risks descriptions. The case study was carried out using parameters like: maximum depth of the QRef, extent of the QRef, share of buildings in the flooding area with protective gear, inhabitants within the hazard prone zone, potential number of casualties for a QRef flooding, number of cultural sites within flooding limits, contaminated sites in the flooding areas, etc. These parameters are subject to change from one case study to another, as they should be tied to the particular features of the study area, the treated hazard, and the availability of the data.

Then, information related to hydrologic and hydraulic studies (flood depth and limits maps for different flooding scenarios), cartographic data on buildings (digital elevation model, housing typologies, buildings floor height relative to the natural ground level, etc.), state-damage functions, socio-economical statistics, etc. has been collected to describe the urban area of FloodedCity, and to acquire the input data that will be used to estimate the sustainability of the three management alternatives under study. Such information has been provided by field studies, experts' surveys, geographical databases, literature, etc. Table 5 presents an extract of the worksheet. In this table, some parameters were assigned three values (low, deterministic

Deterministic values could be average values but do not necessarily equal these values.

This case study exhibits an important source of variability: the input data uncertainty. Sustainability assessments require precise data, but no long-term predictions are without uncertainty. These predictions could gain uncertainty from various sources (e.g. assumptions, data, methods, models) that affect the decision-making process. The content in Table 5 demonstrates the potential incidence of uncertainty. The results of the calculation for “social sustainability” CPI using the lower, deterministic and upper values from the ranges of the parameters within this table are presented in Table 6. These results show how the ranking of alternatives could vary according to the input data. The potential range of the CPI of alternative S2 crosses that of alternative S4. These results suggest that alternatives S2 and S4 might score equally (between -0.03 and 1.17). An uncertainty analysis is necessary to improve the sustainability assessment. However, the core purpose of this paper is not to address uncertainty; further work will be conducted to address this issue. Therefore, the case study was carried out using only one set of data that was assumed to contain the deterministic values for the parameters.

The results from this study are the scored criteria assigned to each alternative. Table 7 presents the obtained CPIs, the ranking of each strategy against each criterion, and the sustainability profiles (on the basis of a twenty-one-point scale). No alternative has technical, and institutional unsustainability. Regarding those criteria, the results show an overall improvement in the performance compared to the reference. Regarding “Economic sustainability” and “Environmental sustainability”, alternatives S2 and S4 are predicted not to be gainful, and environmentally friendly. Figure 4 visually compares the performance of the assessed alternatives. For the integrated decision-making perspective, rankings that depend on the different decision rules are summarised in Table 8. These results illustrate that the most sustainable solution is most likely alternative S3, and that the least sustainable option is alternative S2. Based on the six rules, alternative S3 ranks first six times, while alternative S2 ranks second twice and third three times. By ranking second three times and third two times, alternative S4 seems to be less interesting than alternative S3 but more attractive than alternative S2.

Some subjective methodological choices have been made in this basic assessment process: the use of a twenty-one-point scale, an equal weighting of criteria, and the estimation of the final sustainability performance taking into consideration all the five selected criteria. This ranking may differ if these choices change. Therefore, a sensitivity analysis of the ranking is performed in order to identify influential choices in this case study, and to provide the decision-maker with further insight.

Sensitivity analysis seems fundamental to check if the outcomes of the adopted basic assessment process are robust, or are affected by changes in the length of the rating scale, the weighting factors of criteria, and the number of criteria retained for the sustainability performance index (SPI) calculation.

The case study accounts for some features that are not often considered during the existing natural risk management processes. The findings reveal that the developed framework can provide an excellent informational resource about indicators and criteria that is optimised for each possible management strategy. The results can also highlight the shortfalls in each case. Therefore, based on this supportive analysis tool, decision-makers could compare the sustainability performance of the identified strategies and choose the most sustainable one. They could also monitor the progress toward sustainability or their failure to meet sustainability goals.

As recognised by Helming et al. (2011), “it is fair to say that no methodological framework for ex ante impact assessment will ever manage to comprehensively capture the complex relationships between changes in policy […] and the resulting changes in social, economic, and environmental systems”. No tool will ever be able to capture and reflect every possible impact of a decision. Therefore, this tool does not provide a “set of best parameters, indicators and criteria”; it instead provides a formal and credible set that might be used to support decision-making processes. This paper introduces a methodology prototype that proposes simple calculations through an accessible conceptual approach. These calculations should be appropriate for managing natural risks, and are transferable to any other country.

Decision making for a sustainable risk management policy depends on the distinctive characteristics of each case study. Because the decision-makers weight the parameters, indicators and criteria, the results cannot remain the same when the aggregation framework changes. For a given territory with the same data, sustainability appraisals are liable to evolve because the weights are assigned based on value judgements. In the future and to enhance the robustness of the tool, the variance introduced by the weighting changes regarding the sustainability performance should be subjected to a sensitivity analysis.

Although uncertainty is not the focus of this paper, it is interesting to analyse the impact of the estimated performance range on sustainability profile. Figure 5 presents the shape of the possible values of CPIs of alternative S4. The space for the predicted values is situated between the first and the third graphs along the axes, demonstrating that decisions can vary between these two endpoints and can potentially include the value ranges of other options. Therefore, wider index ranges generate more uncertainty.

Ex ante sustainability assessments, as well as territorial dynamics and various forecasts, contain several assumptions. Nevertheless, these assumptions are unable to reflect the long-term characteristics of hazardous events or effects of the planned decisions because, as noted by Zhu et al. (2011), “the future is inherently uncertain, all exercises about the future are facing and should cope with great uncertainty”. As a strategic appraisal, the sustainability assessment needs to address the inevitable uncertainties introduced during the formulation and implementation processes for the decisions: uncertainty analysis should be an integral phase of the sustainability assessment. In the specific context of risk management, this perspective is confirmed by Olbrich et al. (2009), who argue that “any meaningful assessment of sustainability of risk management strategies faces issues on a fundamental level: the necessity to address uncertainty about the system dynamics in the criterion used for the assessment”. Therefore, developing methodological approaches that can handle uncertainty and examine its effects on the results of a study is critical. Further INCERDD project work is being conducted to develop a framework that accounts for uncertainty within a risk management policy-making process that is geared toward sustainable decisions.