This paper evaluates the efficacy of various cross-validation partitioning methods in capturing spatial dependencies during hyperparameter tuning for landslide spatial prediction (susceptibility). Additionally, it examines the impact of integrating gradient-boosted decision trees with mixed-effect modeling to account for spatial non-stationarity in landslide susceptibility predictions. They use this approach also explore the influence of forest properties on the spatial prediction of landslide occurrence during a 2023 major rainfall event in southern Norway. SHAP values were used to explore relative and rank of variable importance when comparing models with different spatial dependency structures and tuning approaches. Partial dependency plots of SHAP values were used to peak at general landslide response-predictor trends.

Overall, there are some very good methods and process-based findings in this paper that would serve the landslide susceptibility modelling community quite well:

Methods:

• SHAP values and rank can be influenced by the (spatial) partitioning methods used in nested CV for model hyperparameter tuning.
• Accounting for spatial dependence in model residuals does not result in better model performance (but still important for non-biased interpretation of response-prediction relationships in ML models)

Process:

• Tree properties can have a strong influence on landslide predictions

The authors should be cautious not to over-interpret results beyond what their methods can strictly support, in particular how the number of predictors and correlations between them may influence the variable importance results. Furthermore, the current structure of the Results and Discussion sections makes the narrative difficult to follow. Because interpretations are interspersed throughout both sections, it is challenging to identify coherent summaries of the data or clear trends in the authors' explanations.

Major Comments

A major limitation of this paper is the authors’ tendency to overinterpret the results supported by their methods. For example, variable importance ranking is highly variable depending on the collinearity between predictors/variables. The authors often provide conjectures on why variables, like elevation are important, without accounting for possible correlations (e.g. with precipitation and landuse).

Adding in a correlation analysis of the predictor variables can help strengthen the findings or further provide explanation for why variable rankings differ between different model setups. For example, I would guess the importance of elevation and precipitation are flipped between spatial and non-spatial due to them being highly correlated.

Also, adding in a figure/map to illustrate what the different spatial partitioning approaches look like when applied to the landslide data would help in interpreting the results. Especially, to help differentiate if some spatial partitioning approaches are favoring a more ‘locally’ (vs. globally) tuned model due to their spatially coherent groups.

Additionally, the interpretation of the results relies heavily on process-based knowledge. I think the findings would benefit greatly from a discussion that considers the spatial (structure) heterogeneity of the predictors and their potential to differential across space where landslides occur, and how these are captured by the different spatial portioning approaches, and modelling methods.

Minor Comments

L15. Need to provide context to why forest-only dataset is used in the abstract. Currently missing.

L46. Need to be more explicit to what is meant by data-driven. E.g. a physical model can be data-driven if observation data is used for it’s tuning. Maybe empirical models could be used instead.

L58. “almost perfect levels of landslide prediction accuracy” – I would argue there is no perfect prediction accuracy 😊. Maybe word they are highly overoptimistic.

L62. “Machine learning” to general here since later decision tree structure is mentioned in the sentence… SVM’s don’t have decisions trees.

L75. Brenning 2012 in IEEE IGRSS missing from references (original spatial CV paper).

L83. May want to look at Schratz et al 2019 in Ecological Modelling to add to the discussion – they similarly explore spatial partitioning schemes on hyperparameter tuning for predictions with spatial data.

L84. “The majority of recent .. studies appear to apply random or no CV”. The authors’ cannot state this without a reference to a review paper or them doing the review.

L105. Authors state they didn’t create a susceptibility prediction map because it was event specific data. I suggest they do add one to qualitatively assess the geomorphological plausibility of the prediction results, which can provide further evidence to support one method over another.

L115. Goetz et al 2015 in NHESS also assess (regionally) the influence of forest structure on landslide spatial prediction (beyond landuse).

L144-147. I would suggest removing this section. It is vague and doesn’t help explain the methods used.

L149. “The statistical and machine learning”, only ML were applied in this analysis, not statistical.

L160. “aim to establish causal relationships”, SHAP values used in the analysis represent contribution, not causation.

L176. Reference Fig. 7 here to point the reader to what this sampling of control points looks like.

L.333. A figure/map of what they sampling partition groups look like when applied to the landslide and control data would help the reader better understand the results.

L400. Figure 3. If making model comparisons, the x-axis values should be the same between each SHAP value summary plots – e.g. -0.4 to 0.75. This helps makes it easier for the reader to make comparisons.

L.562. “Shallow landslides did not occur in areas with highest rainfall intensities” – the authors interpret this as possible unreliable data. It’s probably more likely that higher rainfall intensity does not equal more landslide frequency. Most landslides will likely be triggered at threshold below the maximum rainfall intensity.

L575. Can bedrock really explain why landslides were triggered in southeastern areas – the SHAP value for bedrock was ~0.?

L585-588. Too much conjecture. Hard to explain the spatial patterns of rainfall act as spatial structure proxies in non-spatial models, especially without out seeing what the spatial partitions for model tuning look like. It could be possible that the non-spatial model looked for more global patterns vs. the spatial methods that may have relied on more local patterns for model tuning (difficult to comment without seeing what the partitions look like.

L603. “flow accumulation threshold of 5000 m2” – we cannot make this statement from the plots, we can say there is a trend around their based on the modeled data, but due to many predictors and likely high correlations, we have to be cautious to over interpret the “bumps” in the partial dependence plots.

L620-625. The authors may be mis-interpreting the results. They suggest the meteorological, vegetation and soil data may be too coarse to be good. It might be true, but it could also be highly correlated with elevation, tree cover or between each other, and thus have a lower ranking, or the parameters are splitting the importance. Hard to tell without a correlation analysis.

L716. Lower ranking of other tree properties may be because they are highly correlated with tree type.

L784. “We have shown that ML results are highly sensitive” – good to be specific here. Only gradient boosting was assessed, so cannot generalize that all ML is sensitive to CV techniques and spatial modeling choices.

The manuscript makes a methodologically relevant contribution to increasing the understanding of the determinants of shallow-landslide/debris flows. Its main focus is the systematic comparison of spatial and non-spatial gradient-boosted models under different cross-validation strategies, using a relatively well-documented event inventory from the 2023 'Hans' storm. The focus on spatial autocorrelation, nested cross-validation, and the instability of predictor-importance rankings is valuable, and the finding that small-block spatial CV offers a pragmatic compromise between predictive performance and residual spatial structure is well supported. It is rare to see results presented so well in a manuscript. The forest-only analysis is also a useful addition, especially because forest characteristics are rarely treated in more detail than broad land-use classes, so this was great to see. Particularly the discussion on quantitative model evaluation vs plausibility was a meaningful component of the manuscript.

Overall, this is a strong hypothesis-driven study that tests specific questions with a useful methodological contribution, and the results deliver important insights concerning different cross-validation choices. I would recommend publication after minor revisions, since a few aspects need clarification before the manuscript is fully publishable. These concern mainly the framing and context, as follows:

1. The first paragraphs of the introduction should be tightened a little. The justification for the study - by situating it in the context of global figures - seems a little off for a Northern-European study, and the text would also benefit from clearer separation between landslide hazard, exposure, impacts/loss. It is unclear whether the authors mean that landslides themselves contribute to increasing losses globally, or whether this is primarily an exposure and climate-change argument.
2. The event inventory and landslide processes require more explanation. Initially, the authors refer to the “Hans” shallow landslide inventory. Then, in L152 the terms debris slides, debris flows, and debris floods are introduced, followed by debris avalanches in L163. I read sections 1 and 2 with shallow landslides in mind, which in my experience have relatively small release areas (100-1000 m2) with reduced run-out length compared to debris flows and depending on the water content. Therefore, the differences of the named processes should be defined more explicitly. The exclusion of debris floods is reasonable, but the authors should explain more clearly how these were identified and why they are not treated as part of the same release-process population. Example photographs or supplementary figures would help. This is important for interpreting and understanding the modelling results correctly, which is currently only possible by accessing the inventory description by Rüther et al., 2024.
3. In my view the control-point sampling strategy needs further justification. The approach is in general innovative and a good one. Limiting controls by elevation, slope, and relative precipitation is defensible, but land use is strongly imbalanced, with very few control points in agricultural land – despite this likely being an important predictor which later had to be removed due to the sample size. I suggest the authors discuss the potential bias more explicitly. While I don't believe a sensitivity test including land-use stratified controls is necessary due to the inclusion of high-resolution woody vegetation products, it would be important to discuss whether the sampling strategy of control points (within the feature space of selected variables) potentially removes otherwise meaningful predictors, or reduces their otherwise expected importance. This was noted in the discussion with respect to slope (L724), but the implication of this sampling approach could also be mentioned in the methods (adding to L175/176).
4. The hydrometeorological predictors need better justification. The use of a 3-day precipitation window follows the storm period, but the authors should explain why this is sufficient without including longer antecedent windows such as 7 or 14 days. Many previous studies have shown the importance of pre-event soil moisture conditions. While the inclusion of groundwater benefits the model, antecedent rainfall/wetness is also mechanistically important to shallow landslide initiation and could have influenced the results. I would recommend testing the effect of including a predictor to represent this influence of antecedent soil moisture in the model.
5. The discussion around soil indicators and tree type is well done, drawing extensively on previous studies. However, the results are event-specific and several variables may act as proxies for unobserved soil, land-use, or mapping effects. While this is mentioned, e.g. in Lines 800-804, the sample size is relatively small which has been shown to influence the effect size of predictors (e.g., https://doi.org/10.1016/j.geomorph.2023.108795). In this context, it would add value to comment on the advantages and disadvantages of event-based (e.g., knowledge of rainfall patterns) vs. multi-temporal inventories (e.g., knowledge of possible changes through time, possible attribution of landslide response to low susceptibility rather than lack of rainfall) for the development of landslide susceptibility models, which are described in the introduction as being urgently needed.

Minor comments:

L31: Additional loading such as rockfalls: Please clarify what is meant by this.

L34-37: As is mentioned with reference to “acceleration of landslide risk”, an increase in the magnitude and frequency of heavy precipitation events requires not only improved susceptibility assessments, but also advances in dynamic hazard and risk assessments. This is because susceptibility is aimed at predicting where landslides are expected to occur in future and not how the frequency and magnitude may change. The relative susceptibility of slopes is unlikely to change significantly with increasing magnitude and frequency of precipitation events. Therefore, future assessments should include rainfall predictors to support dynamic modelling (e.g. https://doi.org/10.1029/2024jf008219), as well as run-out models to support decision-makers in evaluating the risk to infrastructure. I suggest the authors revise this paragraph accordingly.

L88: It’s not immediately clear what is meant by “spatial structure in response data”. Perhaps add an example to clarify, (e.g. what the issue was in the inventory used by Knevels et al., 2023).

L115: It is true that there are limited examples. However, other examples do exist, e.g. https://doi.org/10.1016/j.geomorph.2021.107993, 10.1016/j.geomorph.2023.108870.

L127: Luck is not a valid explanation, so please remove.

L133: I was unable to access Rüther et al., 2024. It would be important to briefly mention what the results of the mapping validation were. 10m resolution imagery in my experience is very coarse for mapping shallow landslide release zones which can be a large as 1 pixel. The imagery may also have implications for mapping in forested areas by obscuring the view beneath canopy. Could this have influenced the results related to tree type?

L146: Rephrase “worked better”. Sounds like the local environment was preferable due to the advanced processing required for shapefiles. I think the final sentence in this paragraph is not needed.