Articles | Volume 22, issue 2
Nat. Hazards Earth Syst. Sci., 22, 539–557, 2022
Nat. Hazards Earth Syst. Sci., 22, 539–557, 2022

Research article 21 Feb 2022

Research article | 21 Feb 2022

Methodological and conceptual challenges in rare and severe event forecast verification

Philip A. Ebert and Peter Milne

Related subject area

Other Hazards (e.g., Glacial and Snow Hazards, Karst, Wildfires Hazards, and Medical Geo-Hazards)
Multi-method monitoring of rockfall activity along the classic route up Mont Blanc (4809 m a.s.l.) to encourage adaptation by mountaineers
Jacques Mourey, Pascal Lacroix, Pierre-Allain Duvillard, Guilhem Marsy, Marco Marcer, Emmanuel Malet, and Ludovic Ravanel
Nat. Hazards Earth Syst. Sci., 22, 445–460,,, 2022
Short summary
Wildfire–atmosphere interaction index for extreme-fire behaviour
Tomàs Artés, Marc Castellnou, Tracy Houston Durrant, and Jesús San-Miguel
Nat. Hazards Earth Syst. Sci., 22, 509–522,,, 2022
Short summary
How is avalanche danger described in textual descriptions in avalanche forecasts in Switzerland? Consistency between forecasters and avalanche danger
Veronika Hutter, Frank Techel, and Ross S. Purves
Nat. Hazards Earth Syst. Sci., 21, 3879–3897,,, 2021
Short summary
Data-based wildfire risk model for Mediterranean ecosystems – case study of the Concepción metropolitan area in central Chile
Edilia Jaque Castillo, Alfonso Fernández, Rodrigo Fuentes Robles, and Carolina G. Ojeda
Nat. Hazards Earth Syst. Sci., 21, 3663–3678,,, 2021
Short summary
The mud volcanoes at Santa Barbara and Aragona (Sicily, Italy): a contribution to risk assessment
Alessandro Gattuso, Francesco Italiano, Giorgio Capasso, Antonino D'Alessandro, Fausto Grassa, Antonino Fabio Pisciotta, and Davide Romano
Nat. Hazards Earth Syst. Sci., 21, 3407–3419,,, 2021
Short summary

Cited articles

Abbe, C.: Unnecessary tornado alarms, Mon. Weather Rev., 27, 255,[255c:UTA]2.0.CO;2, 1899. a
Akosa, J. S.: Predictive accuracy: A misleading performance measure for highly imbalanced data, in: SAS Global Forum 2017, Paper 924, 2–5 April 2017, Orlando, FL, USA, (last access: 6 August 2021), 2017. a
Altman, D. G. and Bland J. M.: Diagnostic tests 3: receiver operating characteristic plots, BMJ, 309, 188,, 1994. a
Anscombe, G. E. M.: Intention, 2nd Edn., Basil Blackwell, Oxford, ISBN 978-0674003996, 1963. a
Benini, R.: Principii di Demografia, in: vol. 29 of Manuali Barbèra di Scienze Giuridiche, Sociali e Politiche, G. Barbèra, Florence, 1901. a
Short summary
There is no consensus about how to assess the quality of binary (yes or no) rare and severe event forecasts, i.e. forecasts involving natural hazards like tornadoes or avalanches. We offer a comprehensive overview of the challenges we face when making such an assessment and provide a critical review of existing solutions. We argue against all but one existing solution to assess the quality of such forecasts and present practical consequences to improve forecasting services.
Final-revised paper