Brief communication: Visualizing uncertainties in landslide susceptibility modeling using bivariate mapping

Schlögl, Matthias; Graser, Anita; Spiekermann, Raphael; Lampert, Jasmin; Steger, Stefan

doi:https://doi.org/10.5194/nhess-2024-213

Preprints

https://doi.org/10.5194/nhess-2024-213

Preprints

03 Dec 2024

| 03 Dec 2024

Status: a revised version of this preprint was accepted for the journal NHESS and is expected to appear here in due course.

Brief communication: Visualizing uncertainties in landslide susceptibility modeling using bivariate mapping

Matthias Schlögl, Anita Graser, Raphael Spiekermann, Jasmin Lampert, and Stefan Steger

Abstract. Effectively communicating uncertainties inherent to statistical models is a challenging yet crucial aspect of the modeling process. This is particularly important in applied research, where output is used and interpreted by scientists and decision makers alike. In disaster risk reduction, susceptibility maps for natural hazards are vital for spatial planning and risk assessment. We present a novel type of landslide susceptibility map that jointly visualizes the estimated susceptibility and the corresponding prediction uncertainty, using an example from a mountainous region in Carinthia, Austria. We also provide implementation guidelines to create such maps using popular free and open-source software packages.

Received: 15 Nov 2024 – Discussion started: 03 Dec 2024

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Matthias Schlögl, Anita Graser, Raphael Spiekermann, Jasmin Lampert, and Stefan Steger

Status: closed

RC1:
'Comment on nhess-2024-213', Anonymous Referee #1, 13 Dec 2024

Using the Central Eastern Alps as an example, this brief communication presents a unique method that integrates the vulnerability and uncertainty of landslides into a single bivariate map. Compared to the conventional approach of examining landslide risk through separate maps, this method offers several advantages. First, by integrating two maps into one, readers can avoid the hassle of cross-referencing, which greatly improves efficiency and reduces the possibility of errors. Secondly, the bivariate map provides a more holistic and intuitive understanding of the complex interplay between vulnerability and uncertainty, enhancing the overall assessment of landslide risk.
The manuscript is well-structured and professionally written. Therefore, the reviewer has no issues with agreeing to publication in its current form.

Citation: https://doi.org/10.5194/nhess-2024-213-RC1
- AC3: 'Reply on RC1', Matthias Schlögl, 29 Jan 2025
  
  Thank you for the positive feedback and for taking the time to review the manuscript. We are glad to hear that you found the manuscript well-structured and professionally written. We appreciate your recommendation for publication.
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC3
RC2:
'Comment on nhess-2024-213', Anonymous Referee #2, 16 Dec 2024

The authors present a bivariate mapping method to spatially visualize both prediction values and uncertainty within the same map. They also provide supplemental material to enable others to apply this approach using free and open-source software packages. The authors discuss and present an approach for estimating uncertainties and classifying susceptibility levels required to build a bivariate map.
This brief communication is well-written and provides a generally significant contribution to the landslide science community, as it highlights a method for improving the communication of uncertainties in hazards predictions. I would recommend minor revisions to address some literature gaps in the introduction and discussion.
The paper fails to acknowledge previously peer-reviewed research on visualization methods for geospatial predictions. The only reference on this topic is from a blog post, which is problematic given that the main contribution of this paper is the visualization of spatial prediction uncertainties. In particular, it is missing references to highly cited research by MacEachren et al (2013 in Cartography and Geographic Information Science), earlier applications to slope stability (Davis and Keller 1997 in Computers & Geosciences), and others who have applied bivariate mapping for communicating spatial prediction and uncertainties (Cola 2013 in Cartography and Geographic Information Science and Nelson 1999 in Cartographic Perspectives).
Another major drawback of this paper is its heavy reliance on reference to works by its own co-authors (Steger and Spiekermann) for landslide susceptibility modelling and data quality, while failing to acknowledge other important contributions in the field.

Other comments:
Introduction
The introduction provides a good overview of methods applied for spatially estimating uncertainties of landslide susceptibility predictions. However, it is lacking background on general methods for visualizing and communicating uncertainties in spatial predictions. Existing research on this topic should be incorporated to help position the authors’ approach within the context of prior work.
L20. The authors rely heavy on citing the co-authors’ prior contributions on data quality. However, there are many different researchers with significant contributions in this field, and these should be acknowledged.
L40. The paper should also reference Heckmann et al. (2014 in NHESS), who used repeated resampling and combined (100) susceptibility maps to estimate the interquartile range (IQR) in spatially predicted probabilities – a similar approach to the one used in this submission.

Methods
Section 2.1
The authors cite a blog post as the source of their methods but fail to reference earlier peer-reviewed contributions using a similar approach (e.g., Cola et al. 2013).
While the authors clarify that their uncertainty calculations are based solely on variations in the sampling of absence (non-landslide) points, given the large number of landslide samples (~2000), they could have also resampled landslides (e.g., using a cross-validation approach). At the very least, they should acknowledge that more robust approaches, which account for variations in landslide presence data, are available.
Section 2.2.1
L87. I think it’s good that you acknowledge the source of your inspiration for your bivariate approach; however, existing peer-review on bivariate mapping approaches to communicate prediction and uncertainty should also be acknowledged.
Results
L133. The authors mention the term “geomorphic plausibility” in the introduction, but don’t define it. I think it would be useful to the reader to define it.
Discussion
L159. “In addition, the type of uncertainty conveyed should be kept in mind” – what are you trying to communicate with this sentence? It’s not clear.
L164. “In landslide susceptibility modeling, results are commonly discretized into three classes signifying low, medium and high susceptibility” – this statement is highly debatable. There are a wide range of approaches in practice to classify landslide susceptibility levels.
L166. The authors again cite only co-authors’ works (Spiekermann and Steger), even though there are other well-cited approaches for calculating breaks (e.g. Chung and Fabbri 2003) including accounting for proportion of landslides covered (e.g. Petschko et al 2014), as applied in this submission.
L175. The authors discuss the importance of color considerations but should expand on how their chosen color palette addresses issues like color impairment (e.g., for colorblind readers). Providing recommendations for alternative palettes or best practices would be helpful for readers.
L185. The authors should reference other approaches for quantifying uncertainties in landslide susceptibility models to provide a more balanced discussion.

Citation: https://doi.org/10.5194/nhess-2024-213-RC2
- AC2: 'Reply on RC2', Matthias Schlögl, 29 Jan 2025
  
  Note: Reviewer comments are typeset in italic and our responses in regular style.
  ## Response to RC2
  The authors present a bivariate mapping method to spatially visualize both prediction values and uncertainty within the same map. They also provide supplemental material to enable others to apply this approach using free and open-source software packages. The authors discuss and present an approach for estimating uncertainties and classifying susceptibility levels required to build a bivariate map.
  This brief communication is well-written and provides a generally significant contribution to the landslide science community, as it highlights a method for improving the communication of uncertainties in hazards predictions. I would recommend minor revisions to address some literature gaps in the introduction and discussion.
  The paper fails to acknowledge previously peer-reviewed research on visualization methods for geospatial predictions. The only reference on this topic is from a blog post, which is problematic given that the main contribution of this paper is the visualization of spatial prediction uncertainties. In particular, it is missing references to highly cited research by MacEachren et al (2013 in Cartography and Geographic Information Science), earlier applications to slope stability (Davis and Keller 1997 in Computers & Geosciences), and others who have applied bivariate mapping for communicating spatial prediction and uncertainties (Cola 2013 in Cartography and Geographic Information Science and Nelson 1999 in Cartographic Perspectives).
  Another major drawback of this paper is its heavy reliance on reference to works by its own co-authors (Steger and Spiekermann) for landslide susceptibility modelling and data quality, while failing to acknowledge other important contributions in the field.
  
  Thanks for the constructive, comprehensive and helpful feedback provided in this reviewer comment.
  We agree that there are some literature gaps in this article. To some extent, the brevity in some aspects is attributable to the manuscript type (i.e., "Brief Communication") and the corresponding guidelines in terms of manuscript length, number of figures and number of references (c.f. <https://www.natural-hazards-and-earth-system-sciences.net/about/manuscript_types.html>). We did already stretch the limits in terms of the number of references, but felt that a more extensive reference list was needed for this article. In our endeavor to keep the article length and reference list short, we have indeed omitted some potentially interesting publications that could be cited here.
  We agree with the reviewer that the papers mentioned are of high relevance and will add references to the following publications as suggested by the reviewer:
  
  - MacEachren et al (2005): <https://doi.org/10.1559/1523040054738936>
  
  - Davis and Keller (1997): <https://doi.org/10.1016/S0098-3004(97)00012-5>
  
  - DeCola (2013): <https://doi.org/10.1559/152304002782008413>
  
  - Nelson (1999): <https://doi.org/10.14714/CP32.625>
  Concerning the references by the co-authors, the article includes one reference to a paper by Spiekermann et al. and indeed four (well cited) references to manuscripts authored by Steger et al. Given the background of the senior author, whose research focus is on uncertainty assessment in landslide susceptibility modelling since many years, we think that this number is not excessive.
  
  ### Other comments
  #### Introduction
  The introduction provides a good overview of methods applied for spatially estimating uncertainties of landslide susceptibility predictions. However, it is lacking background on general methods for visualizing and communicating uncertainties in spatial predictions. Existing research on this topic should be incorporated to help position the authors’ approach within the context of prior work.
  We will add the following references related to visualizing and communicating uncertainties in spatial predictions to the introduction:
  
  - Kinkeldey et al. (2014): <http://doi.org/10.1179/1743277414Y.0000000099>
  
  - MacEachren et al. (2005): <https://doi.org/10.1559/1523040054738936>
  
  - Davis and Keller (1997): <https://doi.org/10.1016/S0098-3004(97)00012-5>
  
  L20. The authors rely heavy on citing the co-authors’ prior contributions on data quality. However, there are many different researchers with significant contributions in this field, and these should be acknowledged.
  The focus on our prior contributions simply reflects our emphasis on data quality within previous research. We agree with your suggestion and have added references to other significant contributions to ensure broader acknowledgment:
  
  - Gaidzik & Ramírez-Herrera (2021): <https://doi.org/10.1038/s41598-021-98830-y>
  
  - Loche et al. (2022): <https://doi.org/10.1016/j.earscirev.2022.104125>
  
  L40. The paper should also reference Heckmann et al. (2014 in NHESS), who used repeated resampling and combined (100) susceptibility maps to estimate the interquartile range (IQR) in spatially predicted probabilities – a similar approach to the one used in this submission.
  Thank you for this literature suggestion. We will add a reference to Heckmann et al. (2014) <https://doi.org/10.5194/nhess-14-259-2014>. We noted that the metric used in this study is actually the interquantile range between the 0.95-quantile and the 0.05-quantile, thereby encompassing 90\% of the modelled susceptibility values.
  
  #### Methods
  Section 2.1: The authors cite a blog post as the source of their methods but fail to reference earlier peer-reviewed contributions using a similar approach (e.g., Cola et al. 2013). While the authors clarify that their uncertainty calculations are based solely on variations in the sampling of absence (non-landslide) points, given the large number of landslide samples (~2000), they could have also resampled landslides (e.g., using a cross-validation approach). At the very least, they should acknowledge that more robust approaches, which account for variations in landslide presence data, are available.
  Thanks for pointing this out. We fully agree and will add the following references in this section:
  
  - Trumbo (1981): <http://doi.org/10.1080/00031305.1981.10479360>
  
  - Nelson (1999): <https://doi.org/10.14714/CP32.625>
  
  - Speich et al. (2015): <http://doi.org/10.1016/j.jhydrol.2015.01.086>
  
  - Teuling et al. (2011): <http://doi.org/10.1002/joc.2153>
  
  - DeCola (2013): <https://doi.org/10.1559/152304002782008413>
  Since the main purpose of this brief communication is on the visualization aspect, we did not discuss the underlying models in detail. We agree that acknowledging that more robust approaches exist is important and will add a corresponding remark in the text.
  
  Section 2.2.1, L87. I think it’s good that you acknowledge the source of your inspiration for your bivariate approach; however, existing peer-review on bivariate mapping approaches to communicate prediction and uncertainty should also be acknowledged.
  Thanks for pointing this out. We fully agree and will update the references in this section, specifically with the references mentioned above.
  
  #### Results
  L133. The authors mention the term “geomorphic plausibility” in the introduction, but don’t define it. I think it would be useful to the reader to define it.
  Thank you for this suggestion. In our previous paper, we introduced the term geomorphic plausibility in the context of landslide susceptibility modelling (cf. Steger et al. 2016). In summary, a geomorphic plausibility assessment serves as an evidence-based approach to identify implausible predictions, akin to the concept of ‘biological plausibility’ used to evaluate empirical relationships in biology. It aims to detect whether a landslide susceptibility map reflects evident errors or artifacts from the classification algorithm or input data, contradicting geomorphic plausibility. Following also Oreskes et al. (1994), a model is geomorphically implausible if the resulting map exhibits detectable flaws. This subjective evaluation is supported by holistic interpretations of exploratory data analyses, modelled relationships, and the spatial structure of predictors and predicted patterns. In order to maintain the focus of this "brief communication type publication," we refer the reader to the mentioned publication by adding, "Geomorphic plausibility evaluation aims to assess whether a landslide susceptibility map aligns with fundamental process knowledge or rather reflects errors stemming from input data or the modeling approach, as detailed in Steger et al. (2016)"
  
  #### Discussion
  L159. “In addition, the type of uncertainty conveyed should be kept in mind” – what are you trying to communicate with this sentence? It’s not clear.
  There are different sources of uncertainty, notably including uncertainties stemming from the landslide inventory, explanatory features or the modelling algorithm used. We propose to expand this paragraph by appending the following sentence to make this more explicit:
  
  "In addition, the type of uncertainty conveyed should be kept in mind: The underlying uncertainties can be aleatoric and epistemic in nature. Especially epistemic uncertainty, stemming from different sources along the modelling workflow, including the inventory, explanatory features or the model used, can be visualized."
  
  L164. “In landslide susceptibility modeling, results are commonly discretized into three classes signifying low, medium and high susceptibility” – this statement is highly debatable. There are a wide range of approaches in practice to classify landslide susceptibility levels.
  We agree and will remove the word 'three', indicating more generally that a discretization of the continuous variable into susceptibility classes is commonly performed.
  
  L166. The authors again cite only co-authors’ works (Spiekermann and Steger), even though there are other well-cited approaches for calculating breaks (e.g. Chung and Fabbri 2003) including accounting for proportion of landslides covered (e.g. Petschko et al 2014), as applied in this submission.
  With respect to the derivation of class intervals we did cite the following publications (L162f): Slocum et al. (2022), Jiang (2013), Jenks and Caspall (1971) as well as Fisher (1958).
  
  We will supplement this paragraph with the following additional references:
  
  - Chung and Fabbri (2003): <http://doi.org/10.1023/B:NHAZ.0000007172.62651.2b>
  
  - Costanzo et al. (2012): <http://doi.org/10.5194/nhess-12-327-2012>
  
  - Conoscenti et al. (2016): <http://doi.org/10.1016/j.geomorph.2016.03.006>
  
  - Petschko et al. (2014): <http://doi.org/10.5194/nhess-14-95-2014>
  
  - Hussin et al. (2016): <http://doi.org/10.1016/j.geomorph.2015.10.030>
  
  L175. The authors discuss the importance of color considerations but should expand on how their chosen color palette addresses issues like color impairment (e.g., for colorblind readers). Providing recommendations for alternative palettes or best practices would be helpful for readers.
  In terms of recommendations for alternative palettes we would like to refer to the R package biscale, which implements a set of bivariate mapping palettes. The palettes provided therein are based on recommended palettes used for map representations. We will add the twelve color palettes provided in biscale as well as map representations with six selected palettes in the appendix. In addition, tools such as <https://colorbrewer2.org> or <https://hclwizard.org> (c.f. also the R package `colorspace`) provide valuable assistance for selecting palettes for users with different types of vision deficiencies. We propose to add a reference to these two tools as a footnote.
  
  L185. The authors should reference other approaches for quantifying uncertainties in landslide susceptibility models to provide a more balanced discussion.
  We will supplement this statement with the following references to provide a broader context:
  
  - Rossi et al. (2010): <http://doi.org/10.1016/j.geomorph.2009.06.020>
  
  - Petschko et al. (2014): <http://doi.org/10.5194/nhess-14-95-2014>
  
  - Brenning et al. (2015): <https://doi.org/10.5194/nhess-15-45-2015>
  
  - Lombardo et al. (2020): <https://doi.org/10.1016/j.earscirev.2020.103318>
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC2
CC1:
'Comment on nhess-2024-213', Kamal Serrhini, 26 Dec 2024

The comment was uploaded in the form of a supplement: https://nhess.copernicus.org/preprints/nhess-2024-213/nhess-2024-213-CC1-supplement.pdf

Citation: https://doi.org/10.5194/nhess-2024-213-CC1
- AC1: 'Reply on CC1', Matthias Schlögl, 29 Jan 2025
  
  Note: Reviewer comments are typeset in italic and our responses in regular style.
  ### 1. Related to susceptibility mapping of landslides
  This paper deals with a method for visualizing landslide susceptibility and associated uncertainty through bivariate mapping (randon forest modeling of landslide susceptibility and associated uncertainty, and bivariate mapping). The model used (random forest) results in the mapping of susceptibility (means) and uncertainty (standard deviations), the combination of which (bivariate mapping) is the subject of this paper.
  
  Landslide susceptibility modeling is based on the “randon forest” method (p. 3 of the article: 2.1 Landslide susceptibility modeling). The implementation of “randon forest” is preceded by the collection, selection and the classification of events (1973 events).
  
  What period/years do these events cover?
  
  These events served as target variable (i.e., training labels): in the matrix used by the model (observations in rows and variables in columns), each event is represented by which parameters / characteristics: probability of occurrence, magnitude, damage, etc.?
  
  It would be interesting to indicate the main determinants: elevation, slope, precipitation... (independent variables: Variation Inflation Factor VIF?) selected and the method used to select them (regression coefficient: Ordinary Least Square or other?) with regard to the dependent variable studied (landslide events).
  
  “A more detailed description of the modeling approach as well as an in-depth discussion focusing on statistical performance and geomorphic plausibility is provided in Schlögl et al. (2024)”
  
  Finally, I propose to better describe both the study site (its dimensions: how many km in length and width, number of pixels in rows and columns, etc.) and the method for modeling landslide susceptibility, this method is apparently considered by another paper (Schlögl et al., 2024) but which is currently being evaluated. The aim is to show that the two maps (susceptibility and uncertainty) are based on a method that is both (statistically) validated and (methodologically) reproducible.
  The landslide hazard is probably linked to other natural hazards (precipitation, freezing and thawing periods, etc.). Can't we talk about a landslide triggered by other hazards (multi-hazard)? Perhaps we're probably getting off the topic of this paper (Schlögl et al., 2024?).
  
  Uncertainty can be taken into account at different stages/steps:
  
  - when calculating susceptibility (if possible), resulting in only a single dataset (instead of two datasets: means and standard deviations) by “random forest” itself?
  
  - in post-processing (if possible), by crossing the two datasets (means and standard deviations: like the coefficient of variation)?
  
  - or by bivariate mapping, as proposed in the article (e.g. crossing the two rasters: Raster 1 of Susceptibility * Raster 2 of Uncertainty)
  Thank you for the input and the suggestions provided. Given that the manuscript under consideration is a "Brief Communication"[^1] we feel that these questions exceed the scope of this type of manuscript. We therefore refer to Schlögl et al. (in press), where the questions raised here are answered, and the study area, data, methods and methodology are described and discussed in detail. The companion paper was accepted in Environmental Earth Sciences on 14 December 2024, and should be available online at https://doi.org/10.1007/s12665-024-12041-y soon. In this manuscript, we decided to summarize the core aspects relevant to obtain a brief overview and general understanding of the data visualized in the bivariate map. While the underlying model is of course important, we wanted to focus on the method of presenting and communicating estimates and associated uncertainty via bivariate mapping, using a recent model as a case study example. This is also beneficial for providing data alongside the code in the supplementary GitHub repository.
  [^1]: https://www.natural-hazards-and-earth-system-sciences.net/about/manuscript_types.html
  
  [^2]: Schlögl, Spiekermann & Steger (in press): Towards a holistic assessment of landslide susceptibility models: Insights from the Central Eastern Alps. Environ. Earth Sci. https://doi.org/10.1007/s12665-024-12041-y.
  
  ### 2. Related to mapping of susceptibility and uncertainty
  “We advocate that bivariate mapping is a straightforward yet sound and effective way to communicate landslide susceptibility and the associated uncertainty.”
  
  How can we show/verify that the bivariate map is more effective in conveying the message than the two initial maps of susceptibility and uncertainty?
  
  It's a question of visual perception and cognitive understanding of the final map by end-users (elected representatives, citizens, tourists), in order to confirm that the final map is more effective (or not).
  We did conduct stakeholder workshops (world café method) with selected expert users from the main target group (including geologists from the Austrian national geological survey, geologists from Austrian federal governments and representatives from disaster relief forces), who provided unstructured feedback for using the map a specific context and for specific tasks. While the users needed some time to become accustomed to the visualization initially, once they internalized the legend the combined map was considered to be more effective. The main reasons were that (1) no switching between map views was required, which makes the joint interpretation less cumbersome (i.e., lower cognitive effort for internalizing the legend once is lower than switching between maps), and (2) the consideration of the second dimension (uncertainty) was considered less likely to be neglected. However, a formalized usability evaluation (such as qualitative usability tests with users or usability expert reviews conducted by usability professionals, or quantitative assessments such as A/B tests) or studies conducted by cognitive psychologists would be required to formally verify the effectiveness of this method. Unfortunately, this is beyond the scope of the study.
  
  We propose to expand the methods and results section accordingly, with a special focus on summarizing the core findings in the results section.
  
  When we consider 3 susceptibility classes and 3 uncertainty classes (3 x 3), we obtain 9 classes or 9 color gradations (bivariate map). If we go to 4 x 4, we'll have 16 color gradations, which makes reading the bivariate map even more & more complex...
  This is true. Arguably, the interpretability when using 4 x 4 classes is lower despite the additional amount of information contained in the map, as the 16 different gradiations are more difficult to distinguish visually. Therefore, our results are based on 9 classes, which can be distinguished comparatively easily when using appropriate color palettes while still providing enough information on the core messages conveyed by the map.
  
  Secondly, the use of the visual variable color (which can be aesthetic and attractive) certainly brings us closer to human visual perception, which is immediately colorful (and in 3D), but what mental realities do the color used represent of the landscape / site? Is it interesting to represent uncertainty in blue gradation color (high level of uncertainty in blue)?
  
  The paper can try to present / propose a second color combinations.
  
  As future development, the definitive choice of color used (gradation in one color for each of susceptibility and uncertainty) can be determined / confirmed ALSO with the help of end-users (students, researchers, laypersons, decision makers).
  The choice of color palette is briefly discussed in the discussion section (line 174ff). We emphasize the importance of accessibility (especially for people with vision deficiencies), the cultural and contextual relevance, and point towards employing a user-centered design process for tailoring visualized information to users.
  
  Modifying the color palette is straightforward from a technical point of view, as only the string specifying the name of the palette has to be changed in R. In QGIS, the color palette can easily be changed via the GUI in the properties of the raster layer. In addition to predefined color palettes, custom palettes could be provided relatively easily. We have created several alternative representations for demonstration purposes and will provide these in the appendix.
  
  The 3D block diagrams in the appendices help to understand the results (visual link between the two variables: slope and high landslide susceptibility-uncertainty).
  We also think that the 3D rayshader snapshots provide additional insights by adding information on the terrain. We will add an interactive rayshader visualization as supplementary material.
  
  Is the scale of variation of susceptibility (between 0 and 1) different (or not) from that of uncertainty (standard deviations)? It is the determination of the limits considered for the creation of the 3 classes in both cases that raises the question here in terms of scales of variation of susceptibility and uncertainty. Has uncertainty been standardized?
  Both scales refer to the same dimensionless quantity, namely landslide susceptibility, proxied by the classification probability of the underlying binary classification problem. Susceptibility is the mean of the ensemble, uncertainty its corresponding standard deviation. We do discuss the issue of deriving univariate class intervals in continuous numerical vectors in the discussion section (line 160 ff).
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC1

Status: closed

RC1:
'Comment on nhess-2024-213', Anonymous Referee #1, 13 Dec 2024

Using the Central Eastern Alps as an example, this brief communication presents a unique method that integrates the vulnerability and uncertainty of landslides into a single bivariate map. Compared to the conventional approach of examining landslide risk through separate maps, this method offers several advantages. First, by integrating two maps into one, readers can avoid the hassle of cross-referencing, which greatly improves efficiency and reduces the possibility of errors. Secondly, the bivariate map provides a more holistic and intuitive understanding of the complex interplay between vulnerability and uncertainty, enhancing the overall assessment of landslide risk.
The manuscript is well-structured and professionally written. Therefore, the reviewer has no issues with agreeing to publication in its current form.

Citation: https://doi.org/10.5194/nhess-2024-213-RC1
- AC3: 'Reply on RC1', Matthias Schlögl, 29 Jan 2025
  
  Thank you for the positive feedback and for taking the time to review the manuscript. We are glad to hear that you found the manuscript well-structured and professionally written. We appreciate your recommendation for publication.
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC3
RC2:
'Comment on nhess-2024-213', Anonymous Referee #2, 16 Dec 2024

The authors present a bivariate mapping method to spatially visualize both prediction values and uncertainty within the same map. They also provide supplemental material to enable others to apply this approach using free and open-source software packages. The authors discuss and present an approach for estimating uncertainties and classifying susceptibility levels required to build a bivariate map.
This brief communication is well-written and provides a generally significant contribution to the landslide science community, as it highlights a method for improving the communication of uncertainties in hazards predictions. I would recommend minor revisions to address some literature gaps in the introduction and discussion.
The paper fails to acknowledge previously peer-reviewed research on visualization methods for geospatial predictions. The only reference on this topic is from a blog post, which is problematic given that the main contribution of this paper is the visualization of spatial prediction uncertainties. In particular, it is missing references to highly cited research by MacEachren et al (2013 in Cartography and Geographic Information Science), earlier applications to slope stability (Davis and Keller 1997 in Computers & Geosciences), and others who have applied bivariate mapping for communicating spatial prediction and uncertainties (Cola 2013 in Cartography and Geographic Information Science and Nelson 1999 in Cartographic Perspectives).
Another major drawback of this paper is its heavy reliance on reference to works by its own co-authors (Steger and Spiekermann) for landslide susceptibility modelling and data quality, while failing to acknowledge other important contributions in the field.

Other comments:
Introduction
The introduction provides a good overview of methods applied for spatially estimating uncertainties of landslide susceptibility predictions. However, it is lacking background on general methods for visualizing and communicating uncertainties in spatial predictions. Existing research on this topic should be incorporated to help position the authors’ approach within the context of prior work.
L20. The authors rely heavy on citing the co-authors’ prior contributions on data quality. However, there are many different researchers with significant contributions in this field, and these should be acknowledged.
L40. The paper should also reference Heckmann et al. (2014 in NHESS), who used repeated resampling and combined (100) susceptibility maps to estimate the interquartile range (IQR) in spatially predicted probabilities – a similar approach to the one used in this submission.

Methods
Section 2.1
The authors cite a blog post as the source of their methods but fail to reference earlier peer-reviewed contributions using a similar approach (e.g., Cola et al. 2013).
While the authors clarify that their uncertainty calculations are based solely on variations in the sampling of absence (non-landslide) points, given the large number of landslide samples (~2000), they could have also resampled landslides (e.g., using a cross-validation approach). At the very least, they should acknowledge that more robust approaches, which account for variations in landslide presence data, are available.
Section 2.2.1
L87. I think it’s good that you acknowledge the source of your inspiration for your bivariate approach; however, existing peer-review on bivariate mapping approaches to communicate prediction and uncertainty should also be acknowledged.
Results
L133. The authors mention the term “geomorphic plausibility” in the introduction, but don’t define it. I think it would be useful to the reader to define it.
Discussion
L159. “In addition, the type of uncertainty conveyed should be kept in mind” – what are you trying to communicate with this sentence? It’s not clear.
L164. “In landslide susceptibility modeling, results are commonly discretized into three classes signifying low, medium and high susceptibility” – this statement is highly debatable. There are a wide range of approaches in practice to classify landslide susceptibility levels.
L166. The authors again cite only co-authors’ works (Spiekermann and Steger), even though there are other well-cited approaches for calculating breaks (e.g. Chung and Fabbri 2003) including accounting for proportion of landslides covered (e.g. Petschko et al 2014), as applied in this submission.
L175. The authors discuss the importance of color considerations but should expand on how their chosen color palette addresses issues like color impairment (e.g., for colorblind readers). Providing recommendations for alternative palettes or best practices would be helpful for readers.
L185. The authors should reference other approaches for quantifying uncertainties in landslide susceptibility models to provide a more balanced discussion.

Citation: https://doi.org/10.5194/nhess-2024-213-RC2
- AC2: 'Reply on RC2', Matthias Schlögl, 29 Jan 2025
  
  Note: Reviewer comments are typeset in italic and our responses in regular style.
  ## Response to RC2
  The authors present a bivariate mapping method to spatially visualize both prediction values and uncertainty within the same map. They also provide supplemental material to enable others to apply this approach using free and open-source software packages. The authors discuss and present an approach for estimating uncertainties and classifying susceptibility levels required to build a bivariate map.
  This brief communication is well-written and provides a generally significant contribution to the landslide science community, as it highlights a method for improving the communication of uncertainties in hazards predictions. I would recommend minor revisions to address some literature gaps in the introduction and discussion.
  The paper fails to acknowledge previously peer-reviewed research on visualization methods for geospatial predictions. The only reference on this topic is from a blog post, which is problematic given that the main contribution of this paper is the visualization of spatial prediction uncertainties. In particular, it is missing references to highly cited research by MacEachren et al (2013 in Cartography and Geographic Information Science), earlier applications to slope stability (Davis and Keller 1997 in Computers & Geosciences), and others who have applied bivariate mapping for communicating spatial prediction and uncertainties (Cola 2013 in Cartography and Geographic Information Science and Nelson 1999 in Cartographic Perspectives).
  Another major drawback of this paper is its heavy reliance on reference to works by its own co-authors (Steger and Spiekermann) for landslide susceptibility modelling and data quality, while failing to acknowledge other important contributions in the field.
  
  Thanks for the constructive, comprehensive and helpful feedback provided in this reviewer comment.
  We agree that there are some literature gaps in this article. To some extent, the brevity in some aspects is attributable to the manuscript type (i.e., "Brief Communication") and the corresponding guidelines in terms of manuscript length, number of figures and number of references (c.f. <https://www.natural-hazards-and-earth-system-sciences.net/about/manuscript_types.html>). We did already stretch the limits in terms of the number of references, but felt that a more extensive reference list was needed for this article. In our endeavor to keep the article length and reference list short, we have indeed omitted some potentially interesting publications that could be cited here.
  We agree with the reviewer that the papers mentioned are of high relevance and will add references to the following publications as suggested by the reviewer:
  
  - MacEachren et al (2005): <https://doi.org/10.1559/1523040054738936>
  
  - Davis and Keller (1997): <https://doi.org/10.1016/S0098-3004(97)00012-5>
  
  - DeCola (2013): <https://doi.org/10.1559/152304002782008413>
  
  - Nelson (1999): <https://doi.org/10.14714/CP32.625>
  Concerning the references by the co-authors, the article includes one reference to a paper by Spiekermann et al. and indeed four (well cited) references to manuscripts authored by Steger et al. Given the background of the senior author, whose research focus is on uncertainty assessment in landslide susceptibility modelling since many years, we think that this number is not excessive.
  
  ### Other comments
  #### Introduction
  The introduction provides a good overview of methods applied for spatially estimating uncertainties of landslide susceptibility predictions. However, it is lacking background on general methods for visualizing and communicating uncertainties in spatial predictions. Existing research on this topic should be incorporated to help position the authors’ approach within the context of prior work.
  We will add the following references related to visualizing and communicating uncertainties in spatial predictions to the introduction:
  
  - Kinkeldey et al. (2014): <http://doi.org/10.1179/1743277414Y.0000000099>
  
  - MacEachren et al. (2005): <https://doi.org/10.1559/1523040054738936>
  
  - Davis and Keller (1997): <https://doi.org/10.1016/S0098-3004(97)00012-5>
  
  L20. The authors rely heavy on citing the co-authors’ prior contributions on data quality. However, there are many different researchers with significant contributions in this field, and these should be acknowledged.
  The focus on our prior contributions simply reflects our emphasis on data quality within previous research. We agree with your suggestion and have added references to other significant contributions to ensure broader acknowledgment:
  
  - Gaidzik & Ramírez-Herrera (2021): <https://doi.org/10.1038/s41598-021-98830-y>
  
  - Loche et al. (2022): <https://doi.org/10.1016/j.earscirev.2022.104125>
  
  L40. The paper should also reference Heckmann et al. (2014 in NHESS), who used repeated resampling and combined (100) susceptibility maps to estimate the interquartile range (IQR) in spatially predicted probabilities – a similar approach to the one used in this submission.
  Thank you for this literature suggestion. We will add a reference to Heckmann et al. (2014) <https://doi.org/10.5194/nhess-14-259-2014>. We noted that the metric used in this study is actually the interquantile range between the 0.95-quantile and the 0.05-quantile, thereby encompassing 90\% of the modelled susceptibility values.
  
  #### Methods
  Section 2.1: The authors cite a blog post as the source of their methods but fail to reference earlier peer-reviewed contributions using a similar approach (e.g., Cola et al. 2013). While the authors clarify that their uncertainty calculations are based solely on variations in the sampling of absence (non-landslide) points, given the large number of landslide samples (~2000), they could have also resampled landslides (e.g., using a cross-validation approach). At the very least, they should acknowledge that more robust approaches, which account for variations in landslide presence data, are available.
  Thanks for pointing this out. We fully agree and will add the following references in this section:
  
  - Trumbo (1981): <http://doi.org/10.1080/00031305.1981.10479360>
  
  - Nelson (1999): <https://doi.org/10.14714/CP32.625>
  
  - Speich et al. (2015): <http://doi.org/10.1016/j.jhydrol.2015.01.086>
  
  - Teuling et al. (2011): <http://doi.org/10.1002/joc.2153>
  
  - DeCola (2013): <https://doi.org/10.1559/152304002782008413>
  Since the main purpose of this brief communication is on the visualization aspect, we did not discuss the underlying models in detail. We agree that acknowledging that more robust approaches exist is important and will add a corresponding remark in the text.
  
  Section 2.2.1, L87. I think it’s good that you acknowledge the source of your inspiration for your bivariate approach; however, existing peer-review on bivariate mapping approaches to communicate prediction and uncertainty should also be acknowledged.
  Thanks for pointing this out. We fully agree and will update the references in this section, specifically with the references mentioned above.
  
  #### Results
  L133. The authors mention the term “geomorphic plausibility” in the introduction, but don’t define it. I think it would be useful to the reader to define it.
  Thank you for this suggestion. In our previous paper, we introduced the term geomorphic plausibility in the context of landslide susceptibility modelling (cf. Steger et al. 2016). In summary, a geomorphic plausibility assessment serves as an evidence-based approach to identify implausible predictions, akin to the concept of ‘biological plausibility’ used to evaluate empirical relationships in biology. It aims to detect whether a landslide susceptibility map reflects evident errors or artifacts from the classification algorithm or input data, contradicting geomorphic plausibility. Following also Oreskes et al. (1994), a model is geomorphically implausible if the resulting map exhibits detectable flaws. This subjective evaluation is supported by holistic interpretations of exploratory data analyses, modelled relationships, and the spatial structure of predictors and predicted patterns. In order to maintain the focus of this "brief communication type publication," we refer the reader to the mentioned publication by adding, "Geomorphic plausibility evaluation aims to assess whether a landslide susceptibility map aligns with fundamental process knowledge or rather reflects errors stemming from input data or the modeling approach, as detailed in Steger et al. (2016)"
  
  #### Discussion
  L159. “In addition, the type of uncertainty conveyed should be kept in mind” – what are you trying to communicate with this sentence? It’s not clear.
  There are different sources of uncertainty, notably including uncertainties stemming from the landslide inventory, explanatory features or the modelling algorithm used. We propose to expand this paragraph by appending the following sentence to make this more explicit:
  
  "In addition, the type of uncertainty conveyed should be kept in mind: The underlying uncertainties can be aleatoric and epistemic in nature. Especially epistemic uncertainty, stemming from different sources along the modelling workflow, including the inventory, explanatory features or the model used, can be visualized."
  
  L164. “In landslide susceptibility modeling, results are commonly discretized into three classes signifying low, medium and high susceptibility” – this statement is highly debatable. There are a wide range of approaches in practice to classify landslide susceptibility levels.
  We agree and will remove the word 'three', indicating more generally that a discretization of the continuous variable into susceptibility classes is commonly performed.
  
  L166. The authors again cite only co-authors’ works (Spiekermann and Steger), even though there are other well-cited approaches for calculating breaks (e.g. Chung and Fabbri 2003) including accounting for proportion of landslides covered (e.g. Petschko et al 2014), as applied in this submission.
  With respect to the derivation of class intervals we did cite the following publications (L162f): Slocum et al. (2022), Jiang (2013), Jenks and Caspall (1971) as well as Fisher (1958).
  
  We will supplement this paragraph with the following additional references:
  
  - Chung and Fabbri (2003): <http://doi.org/10.1023/B:NHAZ.0000007172.62651.2b>
  
  - Costanzo et al. (2012): <http://doi.org/10.5194/nhess-12-327-2012>
  
  - Conoscenti et al. (2016): <http://doi.org/10.1016/j.geomorph.2016.03.006>
  
  - Petschko et al. (2014): <http://doi.org/10.5194/nhess-14-95-2014>
  
  - Hussin et al. (2016): <http://doi.org/10.1016/j.geomorph.2015.10.030>
  
  L175. The authors discuss the importance of color considerations but should expand on how their chosen color palette addresses issues like color impairment (e.g., for colorblind readers). Providing recommendations for alternative palettes or best practices would be helpful for readers.
  In terms of recommendations for alternative palettes we would like to refer to the R package biscale, which implements a set of bivariate mapping palettes. The palettes provided therein are based on recommended palettes used for map representations. We will add the twelve color palettes provided in biscale as well as map representations with six selected palettes in the appendix. In addition, tools such as <https://colorbrewer2.org> or <https://hclwizard.org> (c.f. also the R package `colorspace`) provide valuable assistance for selecting palettes for users with different types of vision deficiencies. We propose to add a reference to these two tools as a footnote.
  
  L185. The authors should reference other approaches for quantifying uncertainties in landslide susceptibility models to provide a more balanced discussion.
  We will supplement this statement with the following references to provide a broader context:
  
  - Rossi et al. (2010): <http://doi.org/10.1016/j.geomorph.2009.06.020>
  
  - Petschko et al. (2014): <http://doi.org/10.5194/nhess-14-95-2014>
  
  - Brenning et al. (2015): <https://doi.org/10.5194/nhess-15-45-2015>
  
  - Lombardo et al. (2020): <https://doi.org/10.1016/j.earscirev.2020.103318>
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC2
CC1:
'Comment on nhess-2024-213', Kamal Serrhini, 26 Dec 2024

The comment was uploaded in the form of a supplement: https://nhess.copernicus.org/preprints/nhess-2024-213/nhess-2024-213-CC1-supplement.pdf

Citation: https://doi.org/10.5194/nhess-2024-213-CC1
- AC1: 'Reply on CC1', Matthias Schlögl, 29 Jan 2025
  
  Note: Reviewer comments are typeset in italic and our responses in regular style.
  ### 1. Related to susceptibility mapping of landslides
  This paper deals with a method for visualizing landslide susceptibility and associated uncertainty through bivariate mapping (randon forest modeling of landslide susceptibility and associated uncertainty, and bivariate mapping). The model used (random forest) results in the mapping of susceptibility (means) and uncertainty (standard deviations), the combination of which (bivariate mapping) is the subject of this paper.
  
  Landslide susceptibility modeling is based on the “randon forest” method (p. 3 of the article: 2.1 Landslide susceptibility modeling). The implementation of “randon forest” is preceded by the collection, selection and the classification of events (1973 events).
  
  What period/years do these events cover?
  
  These events served as target variable (i.e., training labels): in the matrix used by the model (observations in rows and variables in columns), each event is represented by which parameters / characteristics: probability of occurrence, magnitude, damage, etc.?
  
  It would be interesting to indicate the main determinants: elevation, slope, precipitation... (independent variables: Variation Inflation Factor VIF?) selected and the method used to select them (regression coefficient: Ordinary Least Square or other?) with regard to the dependent variable studied (landslide events).
  
  “A more detailed description of the modeling approach as well as an in-depth discussion focusing on statistical performance and geomorphic plausibility is provided in Schlögl et al. (2024)”
  
  Finally, I propose to better describe both the study site (its dimensions: how many km in length and width, number of pixels in rows and columns, etc.) and the method for modeling landslide susceptibility, this method is apparently considered by another paper (Schlögl et al., 2024) but which is currently being evaluated. The aim is to show that the two maps (susceptibility and uncertainty) are based on a method that is both (statistically) validated and (methodologically) reproducible.
  The landslide hazard is probably linked to other natural hazards (precipitation, freezing and thawing periods, etc.). Can't we talk about a landslide triggered by other hazards (multi-hazard)? Perhaps we're probably getting off the topic of this paper (Schlögl et al., 2024?).
  
  Uncertainty can be taken into account at different stages/steps:
  
  - when calculating susceptibility (if possible), resulting in only a single dataset (instead of two datasets: means and standard deviations) by “random forest” itself?
  
  - in post-processing (if possible), by crossing the two datasets (means and standard deviations: like the coefficient of variation)?
  
  - or by bivariate mapping, as proposed in the article (e.g. crossing the two rasters: Raster 1 of Susceptibility * Raster 2 of Uncertainty)
  Thank you for the input and the suggestions provided. Given that the manuscript under consideration is a "Brief Communication"[^1] we feel that these questions exceed the scope of this type of manuscript. We therefore refer to Schlögl et al. (in press), where the questions raised here are answered, and the study area, data, methods and methodology are described and discussed in detail. The companion paper was accepted in Environmental Earth Sciences on 14 December 2024, and should be available online at https://doi.org/10.1007/s12665-024-12041-y soon. In this manuscript, we decided to summarize the core aspects relevant to obtain a brief overview and general understanding of the data visualized in the bivariate map. While the underlying model is of course important, we wanted to focus on the method of presenting and communicating estimates and associated uncertainty via bivariate mapping, using a recent model as a case study example. This is also beneficial for providing data alongside the code in the supplementary GitHub repository.
  [^1]: https://www.natural-hazards-and-earth-system-sciences.net/about/manuscript_types.html
  
  [^2]: Schlögl, Spiekermann & Steger (in press): Towards a holistic assessment of landslide susceptibility models: Insights from the Central Eastern Alps. Environ. Earth Sci. https://doi.org/10.1007/s12665-024-12041-y.
  
  ### 2. Related to mapping of susceptibility and uncertainty
  “We advocate that bivariate mapping is a straightforward yet sound and effective way to communicate landslide susceptibility and the associated uncertainty.”
  
  How can we show/verify that the bivariate map is more effective in conveying the message than the two initial maps of susceptibility and uncertainty?
  
  It's a question of visual perception and cognitive understanding of the final map by end-users (elected representatives, citizens, tourists), in order to confirm that the final map is more effective (or not).
  We did conduct stakeholder workshops (world café method) with selected expert users from the main target group (including geologists from the Austrian national geological survey, geologists from Austrian federal governments and representatives from disaster relief forces), who provided unstructured feedback for using the map a specific context and for specific tasks. While the users needed some time to become accustomed to the visualization initially, once they internalized the legend the combined map was considered to be more effective. The main reasons were that (1) no switching between map views was required, which makes the joint interpretation less cumbersome (i.e., lower cognitive effort for internalizing the legend once is lower than switching between maps), and (2) the consideration of the second dimension (uncertainty) was considered less likely to be neglected. However, a formalized usability evaluation (such as qualitative usability tests with users or usability expert reviews conducted by usability professionals, or quantitative assessments such as A/B tests) or studies conducted by cognitive psychologists would be required to formally verify the effectiveness of this method. Unfortunately, this is beyond the scope of the study.
  
  We propose to expand the methods and results section accordingly, with a special focus on summarizing the core findings in the results section.
  
  When we consider 3 susceptibility classes and 3 uncertainty classes (3 x 3), we obtain 9 classes or 9 color gradations (bivariate map). If we go to 4 x 4, we'll have 16 color gradations, which makes reading the bivariate map even more & more complex...
  This is true. Arguably, the interpretability when using 4 x 4 classes is lower despite the additional amount of information contained in the map, as the 16 different gradiations are more difficult to distinguish visually. Therefore, our results are based on 9 classes, which can be distinguished comparatively easily when using appropriate color palettes while still providing enough information on the core messages conveyed by the map.
  
  Secondly, the use of the visual variable color (which can be aesthetic and attractive) certainly brings us closer to human visual perception, which is immediately colorful (and in 3D), but what mental realities do the color used represent of the landscape / site? Is it interesting to represent uncertainty in blue gradation color (high level of uncertainty in blue)?
  
  The paper can try to present / propose a second color combinations.
  
  As future development, the definitive choice of color used (gradation in one color for each of susceptibility and uncertainty) can be determined / confirmed ALSO with the help of end-users (students, researchers, laypersons, decision makers).
  The choice of color palette is briefly discussed in the discussion section (line 174ff). We emphasize the importance of accessibility (especially for people with vision deficiencies), the cultural and contextual relevance, and point towards employing a user-centered design process for tailoring visualized information to users.
  
  Modifying the color palette is straightforward from a technical point of view, as only the string specifying the name of the palette has to be changed in R. In QGIS, the color palette can easily be changed via the GUI in the properties of the raster layer. In addition to predefined color palettes, custom palettes could be provided relatively easily. We have created several alternative representations for demonstration purposes and will provide these in the appendix.
  
  The 3D block diagrams in the appendices help to understand the results (visual link between the two variables: slope and high landslide susceptibility-uncertainty).
  We also think that the 3D rayshader snapshots provide additional insights by adding information on the terrain. We will add an interactive rayshader visualization as supplementary material.
  
  Is the scale of variation of susceptibility (between 0 and 1) different (or not) from that of uncertainty (standard deviations)? It is the determination of the limits considered for the creation of the 3 classes in both cases that raises the question here in terms of scales of variation of susceptibility and uncertainty. Has uncertainty been standardized?
  Both scales refer to the same dimensionless quantity, namely landslide susceptibility, proxied by the classification probability of the underlying binary classification problem. Susceptibility is the mean of the ensemble, uncertainty its corresponding standard deviation. We do discuss the issue of deriving univariate class intervals in continuous numerical vectors in the discussion section (line 160 ff).
  
  Citation: https://doi.org/10.5194/nhess-2024-213-AC1

Matthias Schlögl, Anita Graser, Raphael Spiekermann, Jasmin Lampert, and Stefan Steger

Viewed

Total article views: 446 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
371	64	11	446	8	8

HTML: 371
PDF: 64
XML: 11
Total: 446
BibTeX: 8
EndNote: 8

Views and downloads (calculated since 03 Dec 2024)

Month	HTML	PDF	XML	Total
Dec 2024	220	46	8	274
Jan 2025	102	13	3	118
Feb 2025	49	5	0	54

Cumulative views and downloads (calculated since 03 Dec 2024)

Month	HTML	PDF	XML	Total
Dec 2024	220	46	8	274
Jan 2025	102	13	3	118
Feb 2025	49	5	0	54

Viewed (geographical distribution)

Total article views: 391 (including HTML, PDF, and XML) Thereof 391 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 20 Feb 2025

Short summary

Communicating uncertainties is a crucial yet challenging aspect of spatial modelling – especially in applied research, where results inform decisions. In disaster risk reduction, susceptibility maps for natural hazards guide planning and risk assessment, yet their uncertainties are often overlooked. We present a new type of landslide susceptibility map that visualizes both susceptibility and associated uncertainty, alongside guidelines for creating such maps using free and open-source software.


Total:	0
HTML:	0
PDF:	0
XML:	0