Articles | Volume 22, issue 2
Research article
21 Feb 2022
Research article |  | 21 Feb 2022

Methodological and conceptual challenges in rare and severe event forecast verification

Philip A. Ebert and Peter Milne

Related subject area

Other Hazards (e.g., Glacial and Snow Hazards, Karst, Wildfires Hazards, and Medical Geo-Hazards)
Characterizing the rate of spread of large wildfires in emerging fire environments of northwestern Europe using Visible Infrared Imaging Radiometer Suite active fire data
Adrián Cardíl, Victor M. Tapia, Santiago Monedero, Tomás Quiñones, Kerryn Little, Cathelijne R. Stoof, Joaquín Ramirez, and Sergio de-Miguel
Nat. Hazards Earth Syst. Sci., 23, 361–373,,, 2023
Short summary
Evaluation of low-cost Raspberry Pi sensors for structure-from-motion reconstructions of glacier calving fronts
Liam S. Taylor, Duncan J. Quincey, and Mark W. Smith
Nat. Hazards Earth Syst. Sci., 23, 329–341,,, 2023
Short summary
Temporal evolution of crack propagation characteristics in a weak snowpack layer: conditions of crack arrest and sustained propagation
Bastian Bergfeld, Alec van Herwijnen, Grégoire Bobillier, Philipp L. Rosendahl, Philipp Weißgraeber, Valentin Adam, Jürg Dual, and Jürg Schweizer
Nat. Hazards Earth Syst. Sci., 23, 293–315,,, 2023
Short summary
A data-driven model for Fennoscandian wildfire danger
Sigrid Jørgensen Bakke, Niko Wanders, Karin van der Wiel, and Lena Merete Tallaksen
Nat. Hazards Earth Syst. Sci., 23, 65–89,,, 2023
Short summary
Equivalent hazard magnitude scale
Yi Victor Wang and Antonia Sebastian
Nat. Hazards Earth Syst. Sci., 22, 4103–4118,,, 2022
Short summary

Cited articles

Abbe, C.: Unnecessary tornado alarms, Mon. Weather Rev., 27, 255,[255c:UTA]2.0.CO;2, 1899. a
Akosa, J. S.: Predictive accuracy: A misleading performance measure for highly imbalanced data, in: SAS Global Forum 2017, Paper 924, 2–5 April 2017, Orlando, FL, USA, (last access: 6 August 2021), 2017. a
Altman, D. G. and Bland J. M.: Diagnostic tests 3: receiver operating characteristic plots, BMJ, 309, 188,, 1994. a
Anscombe, G. E. M.: Intention, 2nd Edn., Basil Blackwell, Oxford, ISBN 978-0674003996, 1963. a
Benini, R.: Principii di Demografia, in: vol. 29 of Manuali Barbèra di Scienze Giuridiche, Sociali e Politiche, G. Barbèra, Florence, 1901. a
Short summary
There is no consensus about how to assess the quality of binary (yes or no) rare and severe event forecasts, i.e. forecasts involving natural hazards like tornadoes or avalanches. We offer a comprehensive overview of the challenges we face when making such an assessment and provide a critical review of existing solutions. We argue against all but one existing solution to assess the quality of such forecasts and present practical consequences to improve forecasting services.
Final-revised paper