Invited perspectives: The ECMWF strategy 2021–2030 challenges in the area of natural hazards
- European Centre For Medium-range Weather Forecasts, Reading/Bologna/Bonn, UK/Italy/Germany
Correspondence: Florian Pappenberger (email@example.com)
The European Centre for Medium-Range Weather Forecasts (ECMWF) mission is to deliver high-quality global medium-range numerical weather predictions and monitoring of the Earth system to its member states. The modelling and forecasting of natural hazards are an important part of this mission. Challenges in this area include the integration of innovative observations into the Earth system; realistic representations of water, energy and carbon cycles; coupling and initialisation of all Earth system components; adequate representation of uncertainties; supporting the development of user-specific products to enable optimal decision-making under uncertainties; and advances in software engineering. The new ECMWF strategy identified three pillars to sustain its future development (ECMWF, 2021a): science and technology (world-leading weather and Earth system science, cutting-edge technology and computational science), impact (high-quality products fit for purpose, efficient and easy access to products), and people (inspiring and hiring the best experts). Progress in all these areas will need enhanced collaboration with member states and partners across Europe and beyond.
The forecasting of weather, hydrology, the cryosphere and oceans and the representation of past and future climates are both key elements in the understanding and prediction of natural hazards. Forecasts of the Earth system are important to prepare for humanitarian disasters resulting from, for example, tropical cyclones (Emerton et al., 2020; Magnusson et al., 2019), floods (Lavers et at 2020), extreme precipitation events (Lavers et al., 2018) or heat waves (Napoli et al., 2020). Representations of the past climate, known as reanalysis, are key to establishing risk, i.e. determining return periods (Harrigan et al., 2020a) and natural and artificial trends (Zsótér et al., 2020) as well as understanding scientific challenges. Progress in these areas can only be achieved through an Earth system approach (Harrigan et al., 2020b).
The European Centre for Medium-Range Weather Forecasts (ECMWF) vision is to produce cutting-edge science, world-leading weather predictions and monitoring of the Earth system with a particular focus on the medium and sub-seasonal forecast range. To meet this vision, there are several research challenges that need addressing. The required step changes in science and forecasting can be grouped into three pillars (ECMWF, 2021a): science and technology (world-leading weather and Earth system science, cutting-edge technology and computational science), impact (high-quality products fit for purpose, efficiently and easily accessed), and people (inspiring and hiring the best experts). ECMWF has been a collaborative organisation from the start, and the size of the upcoming challenges make it even more important for the organisation to collaborate effectively with its member states.
Earth system model predictions can be substantially improved if we maximise the use of current observations (Fig. 1), e.g. using all available information from existing satellites over land, snow and sea ice in cloudy, rainy and clear conditions (Geer et al., 2018). Newly available observations from satellites and radar will need to be efficiently integrated into the Earth system model. For example, the new EUMETSAT MTGs will provide high-frequency atmospheric profiles as well as real-time lightning detection (lightnings can cause fires). The European Commission's Copernicus Sentinel-6 Michael Freilich (a radar altimeter satellite) will measure sea level change and river levels, addressing a range of natural hazards such as floods, storm surges and droughts. In addition, the operational use of innovative new observing systems, e.g. derived through the Internet of things (IoT, including observations from cars and mobile phones), provides the potential to significantly advance science and forecasts of the Earth system and natural hazards. Collaboration with the ECMWF member states in this area will be particularly useful, as the use of IoT is expected to benefit short-range forecasts even before medium-range ones.
These innovations must be paired with a consistent modelling approach which includes realistic representations of the water, energy and carbon cycles. This is currently not the case for the water cycle; for example, in current data assimilation/forecasts, the mass in the system does not remain constant (principle of mass conservation; Zsótér et al., 2020). The efficient coupling of many of these different processes poses a significant research challenge, closely linked to problems such as the initialisation of a complex Earth system consisting of components like land, snow, rivers, oceans and atmospheric compositions. For instance, the provision of the initial conditions of the snowpack on a global scale, which is important for flood forecasts, can only be addressed with more real-time adequate measurements as well as with a better physical representation of the processes in the snowpack (ECMWF currently utilises a single-layer snow model).
Another essential aspect of weather forecasting is to capture and represent uncertainties by using ensemble predictions, but finite computing and requirements to produce timely forecasts only allow a limited number of ensemble members. A chaotic, flow-dependent system will always have to rely on representing uncertainties within some sort of Monte Carlo-type framework. More research is needed to better represent uncertainties and, for example, to define the optimal ensemble sizes to adequately represent tails of the climate and forecast distribution. Other methods such as post-processing or artificial intelligence (AI) will be able to represent such uncertainties to some degree, and a careful balance between various methods has to be found.
Such scientific advances must be supported by substantial technological innovations. Novel HPC architectures for computing and storage are required to improve the representation of processes and uncertainties. At the same time, existing software code and algorithms need to be adapted to take advantage of these new resources and optimise speedup and scaleup of parallel processing. This requires significant investment in computational science, for example in the use of high-performance, heterogeneous GPU/CPU architectures through domain-specific languages, which will allow domain experts (such as natural hazard scientists) to develop programs independently from the underlying architecture (Mernik et al., 2005). Novel architectures (i.e. GPU, FGPU, etc.) are particularly suited to being used by AI and machine learning (ML), which will play an increasingly revolutionary role throughout the entire research and operational chain, from the quality control of observations and approximation of physical equations, to speeding up execution, and to the improved representation of uncertainties in extreme natural hazards through post-processing (Duben et al., 2021). This is another area where collaboration with the member states will be fruitful. Finally, we will investigate novel ways to address the need for increasingly computationally intensive forecasts and simulations, including computing solutions beyond supercomputers. For example, the OpenIFS@home project allows scientists to exploit spare computational cycles on desktop computers provided by volunteers to run huge ensembles offering researchers a new tool to study weather-forecast-related questions (Sparrow et al., 2021).
The data produced by these sophisticated systems can however be of use only if they are easily accessible and provided reliably. Future Earth system models and forecasts will generate such large volumes of data that the only feasible way to extract relevant information will be to bring computing to the data rather than to transfer the data to a computing infrastructure (Pappenberger and Palkovic, 2020). This also requires research into novel ways of storing and manipulating big data (Hanley et al., 2020). A typical workflow for a natural hazard forecast or model contains the following components: pre-processing, numerical simulation, post-processing and visualisation. Numerical simulations are often run on a high-performance computer, whilst the other tasks are more suited to being executed in a cloud environment, although these different technologies will converge in the future. Moving the computation to the data will have the additional benefit of providing an environment that allows systems and workflow components to be co-designed and shared with and by the member states, stimulating the creation of an eco-system of meteorological data and applications. Research into the efficient management and orchestration of workflows spanning these different computing environments will crucially improve the overall performance of a forecast and model chain. This will allow a larger proportion of limited human and computing resources to be used to improve natural hazard forecasts and models.
Advances in science and technology are only relevant within the context of an operational framework if users can maximise the usefulness and accessibility of high-quality products and outputs. Communication and decision-making using big data representing uncertainties remains a significant challenge which will need to be addressed to increase the impact of natural hazard predictions and reanalysis (Neumann et al., 2018; Thielen-del Pozo et al., 2020). ECMWF will aim to provide detailed Earth system simulations of the past, present and future. Although they may be used for different purposes, simulations at different timescales need to be considered in a holistic way. For instance, reanalysis data may be used to establish return periods, which in turn are used to detect weather time anomalies in forecasting or to make risk-based decisions on climate projections, or they may be used simply on their own for a risk analysis. Particular attention will be devoted to extreme events several weeks ahead to provide skilful predictions of extreme temperature anomalies and hydrological impacts such as droughts and floods, air quality, fires, heat and cold waves, as well as outputs which monitor the environment (i.e. anthropogenic CO2 emissions). Forecasts of natural hazards beyond 2 weeks require an improved understanding of the sources of predictability and teleconnections (Mastrantonas et al., 2020). In this context, it is necessary to focus strongly on a user-oriented evaluation over multiple temporal scales, variables and coupled components which will inform the optimal pathway of model development. Indeed, user feedback especially from the member states needs to be more closely integrated into the evaluation chain to allow for cost–benefit assessments and ensure that the proportion of resources employed in all elements of the system provides maximum benefits. Policy- and decision-makers, emergency responders in the member states and beyond, and the public in general will be able to significantly improve their decision-making when trust in forecasts is increased through metrics which are understandable and meaningful to them (WMO, 2021; Ebert et al., 2018). A key research challenge is to generate detailed and high-resolution Earth system scenarios able to simulate the effects of different environmental policies and sustainable development plans to derive optimal conclusions (European Commission, 2021). Each component of the Earth system and other factors such as environmental policies have uncertainties attached to them which will interact in complex non-linear ways. Therefore, it is important to fully propagate and cascade all dominant uncertainties through the entire value chain, which is key in supporting efficient user decision-making.
Expert and highly motivated people are the most important resource for research or operational activities to improve natural hazard forecasts and models. Working environments are rapidly changing under the pressure and mutual influences of technological advances; globalisation; and, in the last year, the effects of lockdowns. Successful organisations that aim to create step changes to advance the science and practice in natural hazards need to adapt and harness the opportunities that more dynamic working conditions bring. The expansion of ECMWF's activities into the area of computational science and AI for numerical weather prediction also means that the organisation needs to position itself and become attractive in a highly competitive and rapidly changing job market, which is key to being able to address the challenges mentioned above. ECMWF is becoming a multi-site organisation and as such will be more attractive to potential employees and existing staff as it allows for more flexibility in terms of work places while aiming to maintain a “One ECMWF” culture. In parallel, ECMWF will continue to modernise its social policies with particular attention paid to fostering an appropriate work–life balance, e.g. via flexible working patterns. ECMWF is already an organisation with a strong environmental consciousness as it operates the Copernicus Climate Change Service, which makes it an attractive employer. It is therefore essential to continue enhancing environmental awareness, reducing the carbon footprint and embedding environmental awareness even deeper in the organisational culture.
Importantly, all this can be achieved by ECMWF only via a collaborative approach with the European Meteorological Infrastructure as well as with other international and national partners and the private sector.
ECMWF has embarked on a new strategy aiming to produce cutting-edge science and world-leading weather predictions and monitoring of the Earth system in close collaboration with the members of the European Meteorological Infrastructure, for a safe and prosperous society. An essential part of this strategy is to deliver forecasts of high-impact weather events and natural hazards at least 2 weeks in advance. For example, our target is to provide skilful predictions of extreme temperature anomalies and hydrological impacts such as droughts up to 3 weeks ahead on average. We will need to develop a high-precision digital model of our planet that will make it possible to interactively explore various natural processes and human activities. Advancements in machine learning need to be intertwined with physically based models so that they become indistinguishable from each other, revolutionising the European capability to monitor our changing planet and predict events, based on the integration of extreme-scale computing and the real-time exploitation of all available environmental data.
In addition, ECMWF aims to support the natural hazard domain through global reanalyses and re-forecasts of weather and environmental hazards to monitor changing patterns and the predictability of high-impact events. For example, we will develop anthropogenic CO2 emission estimation systems at global, regional and local scales with a full representation of uncertainties to support the implementation of international agreements such as the Paris agreement.
The achievement of this vision will require the adoption of new technologies, the integration of different areas of research, and extensive collaboration with national hydro-meteorological services and other environmental organisations in Europe and worldwide.
The underlying AMSU data which have been used to generate Fig. 1 can be accessed at https://doi.org/10.7289/V53R0QXD (Ferraro et al., 2016). AMSU-A data usage is updated in real time and available publicly from the ECMWF website https://www.ecmwf.int/en/forecasts/quality-our-forecasts/monitoring-observing-system (ECMWF, 2021b). Licenses to access research experiments used to calculate the FSOI can be obtained here: https://www.ecmwf.int/en/forecasts/accessing-forecasts (ECMWF, 2021c).
The authors jointly wrote the manuscript.
The authors declare that they have no conflict of interest.
Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the special issue “Perspectives on challenges and step changes for addressing natural hazards”. It is not associated with a conference.
We would like to thank the reviewers and the editor who helped us to significantly improve the manuscript. We also would like to thank all past, present and future ECMWF staff, collaborators in our Member and Co-operating states and researchers across the globe which contribute to the success of ECMWF.
This paper was edited by Uwe Ulbrich and reviewed by two anonymous referees.
Di Napoli, C., Barnard, C., Prudhomme, C., Cloke, H. L., and Pappenberger, F.: ERA5-HEAT: A global gridded historical dataset of human thermal comfort indices from climate reanalysis, Geosci. Data J., 8, 2–10, https://doi.org/10.1002/gdj3.102, 2020.
Düben, P., Modigliani, U., Geer, A., Siemen, S., Pappenberger, F., Bauer, P., Brown, A., Palkovic, M., Raoult, B., Wedi, N., and Baousis, V: Machine learning at ECMWF: A roadmap for the next 10 years, ECMWF Technical memorandum 878, https://doi.org/10.21957/ge7ckgm, 2021.
Ebert, E., Brown, B., Göber, M., Haiden, T., Mittermaier, M., Nurmi, P., Wilson, L., Jackson, S., Johnston, P., and Schuster, D.: The WMO challenge to develop and demonstrate the best new user-oriented forecast verification metric, Meteorol. Z., 27, 435–440, https://doi.org/10.1127/metz/2018/0892, 2018.
ECMWF: The ECMWF Strategy 2021–2030, available at: https://www.ecmwf.int/en/about/what-we-do/strategy, last access: 22 January 2021a.
ECMWF: Monitoring of the observing system, available at: https://www.ecmwf.int/en/forecasts/quality-our-forecasts/monitoring-observing-system, last access: 13 July 2021b.
ECMWF: Access to forecasts, available at: https://www.ecmwf.int/en/forecasts/accessing-forecasts, last access: 13 July 2021c.
Emerton, R., Cloke, H., Ficchi, A., Hawker, L., de Wit, S., Speight, L., Prudhomme, C., Rundell, P., West, R., Neal, J., Cuna, J., Harrigan, S., Titley, H., Magnusson, L., Pappenberger, F., Klingaman, N., and Stephens, L.: Emergency flood bulletins for Cyclones Idai and Kenneth: a critical evaluation of the use of global flood forecasts for international humanitarian preparedness and response, Int. J. Disast. Risk Re., 50, 101811, https://doi.org/10.1016/j.ijdrr.2020.101811, 2020.
European Commission: available at: https://ec.europa.eu/digital-single-market/en/destination-earth-destine, last access: 22 January 2021.
Ferraro, R. R., Meng, H., Yang, W., and NOAA CDR Program: NOAA Climate Data Record (CDR) of Advanced Microwave Sounding Unit (AMSU)-A, Version 1.0, NOAA National Centers for Environmental Information, https://doi.org/10.7289/V53R0QXD, 2016.
Geer, A. J., Lonitz, K., Weston, P., Kazumori, M., Okamoto, K., Zhu, Y., Huichun Liu, E., Collard, A., Bell, W., Migliorini, S., Chambon, P., Fourrié, N. Kim,, M.-J., Köpken-Watts, C., and Schraff, C.: All-sky satellite data assimilation at operational weather forecasting centres, Q. J. Roy. Meteor. Soc., 144, 1191–1217, https://doi.org/10.1002/qj.3202, 2018.
Hanley, J., Siemen, S., Hawkes, J., Vuckovic, M., Quintino, T., and Pappenberger, F.: Building Cloud-Based Data Services to Enable Earth Science Workflows, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-10297, https://doi.org/10.5194/egusphere-egu2020-10297, 2020.
Harrigan, S., Zsoter, E., Alfieri, L., Prudhomme, C., Salamon, P., Wetterhall, F., Barnard, C., Cloke, H., and Pappenberger, F.: GloFAS-ERA5 operational global river discharge reanalysis 1979–present, Earth Syst. Sci. Data, 12, 2043–2060, https://doi.org/10.5194/essd-12-2043-2020, 2020a.
Harrigan, S., Cloke, H., and Pappenberger, F.: Innovating global hydrological prediction through an Earth system approach, WMO Bulletin, 69, ISSN 0042-9767, 2020b.
Lavers, D. A., Rodwell, M. J., Richardson, D. S., Ralph, F. M., Doyle, J. D., Reynolds, C. A., Tallapragada, V., and Pappenberger, F.: The Gauging and Modeling of Rivers in the Sky, Geophys. Res. Lett., 15, 7828–7834, https://doi.org/10.1029/2018GL079019, 2018.
Lavers, D. A., Ramos, M.-H., Magnusson, L., Pechlivanidis, I., Klein, B., Prudhomme, C., Arnal, L., Crochemore, L., Van Den Hurk, B., Weerts, A. H., Harrigan, S., Cloke, H. L., Richardson, D. S., and Pappenberger, F.: A Vision for Hydrological Prediction, Atmosphere, 11, 237, https://doi.org/10.3390/atmos11030237, 2020.
Mastrantonas, N., Herrera-Lormendez, P., Magnusson, L., Pappenberger, F., and Matschullat, J.: Extreme precipitation events in the Mediterranean: Spatiotemporal characteristics and connection to large-scale atmospheric flow patterns, Int. J. Climatol., 41, 2710–2728, https://doi.org/10.1002/joc.6985, 2020.
Mernik, M., Heering, J., and Sloane, A. M.: When and how to develop domain-specific languages, ACM Comput. Surv., 37, 316–344, https://doi.org/10.1145/1118890.1118892, 2005.
Neumann, J. L., Arnal, L., Emerton, R. E., Griffith, H., Hyslop, S., Theofanidi, S., and Cloke, H. L.: Can seasonal hydrological forecasts inform local decisions and actions? A decision-making activity, Geosci. Commun., 1, 35–57, https://doi.org/10.5194/gc-1-35-2018, 2018.
Pappenberger, F. and Palkovic, M.: Progress towards a European Weather Cloud, ECMWF Newsletter, 165, 24–27, https://doi.org/10.21957/9pft4uy055, 2020.
Sparrow, S., Bowery, A., Carver, G. D., Köhler, M. O., Ollinaho, P., Pappenberger, F., Wallom, D., and Weisheimer, A.: OpenIFS@home version 1: a citizen science project for ensemble weather and climate forecasting, Geosci. Model Dev., 14, 3473–3486, https://doi.org/10.5194/gmd-14-3473-2021, 2021.
Thielen-del Pozo, J., Autogena, L., Portway, J., and Pappenberger, F.: The science-policy interface – in need for “society” as a third component, EGU General Assembly 2020, Online, 4–8 May 2020, EGU2020-21623, https://doi.org/10.5194/egusphere-egu2020-21623, 2020.
WMO: WMO project on High Impact Weather, available at: http://hiweather.net/Lists/15.html, last access: 20 April 2021.
Zsótér, E., Cloke, H. L., Prudhomme, C., Harrigan, S., de Rosnay, P., Munoz-Sabater, J., and Stephens, E.: Trends in the GloFAS-ERA5 river discharge reanalysis, ECMWF Technical Memoranda, 871, Technical Report, ECMWF, https://doi.org/10.21957/p9jrh0xp, 2020.