|This is my 2nd round review of the manuscript "Predicting power outages caused by extratropical storms" by Tervo et al. I highly appreciate the comprehensive revisions performed by the authors. I think that they handled all of the reviewer comments as well as the short comments issued during the interactive discussion appropriately. The manuscript improved in all aspects that were critized in the first review. I specifically appreciate that the authors now provide a little more background information regarding the different classification algorithms and that they included a sensitivity analysis, illustrating the relevance of individual parameters for the best performing algorithm. I hence recommend publication of this manuscript. However, I have a few comments regarding some answers they made to my previous issues. These don't necessarily need to be taken up by the authors in terms of revising the manuscript but I would like to ask the authors to think and respond on these points.|
1. When answering to my previous general comment (e), the authors state:
"[...] The uncertainty would originate as a probabilistic prediction of the classification model, which describes the confidence of the model prediction instead of the reliability of the actual predictions. In other words, the uncertainty would not consider any sources of errors not introduced to the model. For example, the amount of leaves in the trees significantly affects the number of caused outages, but are not considered in the prediction due to shortcomings in available data. The model could predict an incorrect class with a very high confidence as it is not aware of tree leaves at all. [...]".
All of the above is correct but the conclusion of the authors is wrong. In fact it is state-of-the-art and common practice in weather and climate prediction to check if a forecast system's uncertainty is in line with the forecast error. And when it is about probabilistic (ensemble based) forecasts this applies also to the reliability of the probabilistic prediction. In case of a "perfectly reliable" prediction system, the predicted probability matches the average (climatological) occurence of the specific event for all cases when the event was forecasted. And there are proper verification scores that can test for these characteristics. So there are ways to check that beforehand and probabilistic prediction systems that are not perfectly reliable in that sense can usually be corrected via calibration.
As written before: I do not expect that the authors revise their whole approach. I am fine with the comment they added to the manuscript in this respect. I just wanted to clarify this here.
2. When answering to my previous specific comment (11), the authors state:
"[...] As the referee suggests, using specific quantiles would be a proficient way to determine the correct thresholds. However, with an object-based approach, the use of quantiles is not a straightforward task since the object needs to have the same absolute value inside the application domain to be a valid polygon. Therefore, the thresholds of the objects can not be always selected optimally. [...]"
This is a rather cheap excuse. It would be no problem to advance the identification of the storm objects in a sense that not a fixed threshold is used for all grid-boxes but instead a 2D-field containing individual thresholds for the respective grid boxes, still assigning simple boolean values to the resulting object field. In theory even 3D would be possible to account for the seasonality that was mentioned by the authors themselves.
I would like to make the authors aware of the fact that such an algorithm for identifying and tracking of storm fields based on a 2D-quantile-field already exists: The approach that was first introduced by Leckebusch et al. (2008) and since then used in multiple studies on extra-tropical storms (and recently even for tropical storms) is based on exceedances of the local climatological 98th percentile of surface-near wind speed. A detailed description of the complete algorithm is available only from my own PhD-thesis (Kruschke, 2015) but this general principle is named in all papers (in the order of 20-30 or so) based on this algorithm.
I am not fishing for any citations here. and I would name other algorithms containing such a feature if I knew some doing so. It is fine with me that the authors proceed with their algorithm as it is, given that they included a hint towards quantiles being a possibility for optimization. I just wanted to make clear that such a modification is not impossible as indicated by the authors in their response.
3. Closely related to the above, when the authors reply to my specific comment (33), they state:
" [...] Showing objects outside of Finland, for example, the Baltic Sea provides valuable information nevertheless to the operators in the form of preliminary information about approaching storms. The particular message in those cases is: The storm as it is now, would be (or would not be) hazardous to our power network if it was in our region. This gives the operator more tools and time to prepare. [...] "
In principle I agree but I expect that using a fixed threshold results into identifying comparably many storm objects over the Baltic Sea that disappear, once they reach land areas when surface winds are decelerated due to the higher surface roughness, orography and so on. In that case the statement of the authors that such a storm would cause damage over land is not really accurate and (assuming that my expectation is right) frequent warnings regarding storms over the Baltic without any further classification of severity may even lead to ignorance of such cases by power network operators. I don't see how the current algorithm is able to distinguish between "really" dangerous storms approaching over the Baltic Sea and those that can be expected to yield wind speeds over 15 m/s only over the sea, not over land.
Leckebusch et al. (2008): Development and Application of an Objective Storm Severity Measure for the Northeast Atlantic Region. Meteor. Z., 2008, 17, 575–587, DOI: https://doi.org/10.1127/0941-2948/2008/0323
Kruschke, T. (2015): Winter wind storms: Identification, verification of decadal predictions, and regionalization. Institute of Meteorology – Dept. of Earth Sciences – Freie Universität Berlin, Berlin, Germany. URL: https://refubium.fu-berlin.de/handle/fub188/11913