Articles | Volume 22, issue 2
https://doi.org/10.5194/nhess-22-539-2022
https://doi.org/10.5194/nhess-22-539-2022
Research article
 | 
21 Feb 2022
Research article |  | 21 Feb 2022

Methodological and conceptual challenges in rare and severe event forecast verification

Philip A. Ebert and Peter Milne

Related subject area

Other Hazards (e.g., Glacial and Snow Hazards, Karst, Wildfires Hazards, and Medical Geo-Hazards)
How hard do avalanche practitioners tap during snow stability tests?
Håvard B. Toft, Samuel V. Verplanck, and Markus Landrø
Nat. Hazards Earth Syst. Sci., 24, 2757–2772, https://doi.org/10.5194/nhess-24-2757-2024,https://doi.org/10.5194/nhess-24-2757-2024, 2024
Short summary
A large-scale validation of snowpack simulations in support of avalanche forecasting focusing on critical layers
Florian Herla, Pascal Haegeli, Simon Horton, and Patrick Mair
Nat. Hazards Earth Syst. Sci., 24, 2727–2756, https://doi.org/10.5194/nhess-24-2727-2024,https://doi.org/10.5194/nhess-24-2727-2024, 2024
Short summary
A glacial lake outburst flood risk assessment for the Phochhu river basin, Bhutan
Tandin Wangchuk and Ryota Tsubaki
Nat. Hazards Earth Syst. Sci., 24, 2523–2540, https://doi.org/10.5194/nhess-24-2523-2024,https://doi.org/10.5194/nhess-24-2523-2024, 2024
Short summary
AutoATES v2.0: Automated Avalanche Terrain Exposure Scale mapping
Håvard B. Toft, John Sykes, Andrew Schauer, Jordy Hendrikx, and Audun Hetland
Nat. Hazards Earth Syst. Sci., 24, 1779–1793, https://doi.org/10.5194/nhess-24-1779-2024,https://doi.org/10.5194/nhess-24-1779-2024, 2024
Short summary
Modelling the vulnerability of urban settings to wildland–urban interface fires in Chile
Paula Aguirre, Jorge León, Constanza González-Mathiesen, Randy Román, Manuela Penas, and Alonso Ogueda
Nat. Hazards Earth Syst. Sci., 24, 1521–1537, https://doi.org/10.5194/nhess-24-1521-2024,https://doi.org/10.5194/nhess-24-1521-2024, 2024
Short summary

Cited articles

Abbe, C.: Unnecessary tornado alarms, Mon. Weather Rev., 27, 255, https://doi.org/10.1175/1520-0493(1899)27[255c:UTA]2.0.CO;2, 1899. a
Akosa, J. S.: Predictive accuracy: A misleading performance measure for highly imbalanced data, in: SAS Global Forum 2017, Paper 924, 2–5 April 2017, Orlando, FL, USA, http://support.sas.com/resources/papers/proceedings17/0942-2017.pdf (last access: 6 August 2021), 2017. a
Altman, D. G. and Bland J. M.: Diagnostic tests 3: receiver operating characteristic plots, BMJ, 309, 188, https://doi.org/10.1136/bmj.309.6948.188, 1994. a
Anscombe, G. E. M.: Intention, 2nd Edn., Basil Blackwell, Oxford, ISBN 978-0674003996, 1963. a
Benini, R.: Principii di Demografia, in: vol. 29 of Manuali Barbèra di Scienze Giuridiche, Sociali e Politiche, G. Barbèra, Florence, 1901. a
Download
Short summary
There is no consensus about how to assess the quality of binary (yes or no) rare and severe event forecasts, i.e. forecasts involving natural hazards like tornadoes or avalanches. We offer a comprehensive overview of the challenges we face when making such an assessment and provide a critical review of existing solutions. We argue against all but one existing solution to assess the quality of such forecasts and present practical consequences to improve forecasting services.
Altmetrics
Final-revised paper
Preprint