TRPGs are the original form of role-playing games and in principle they are conducted through discussion within an interactive and collaborative storytelling system (Aylett et al., 2008). The players act out the roles of characters and through the game they decide the actions of their character given certain rules and responsibilities within an hypothetical setting (Cover, 2010). In the ANYCaRE game, participants are invited to play specific characters of the decision-making chain (i.e. forecasters, emergency managers or public) in an interactive story related to a weather hazard in a European context. As a narrative-based game, ANYCaRE consists of a storyline and the simulations to be played through various roles. Table 1 describes the key components (storyline, roles, simulations) necessary for the definition of the scenarios and summarizes the questions that guided our design choices and further considerations for each of those components. This table may serve as a design guide for future developers of educational or training exercises in the domain of natural hazard management. For instance, it could help developers to consider options with respect to choices in the real world vs. virtual territory or the time frame and pace of the decision-making related to the type of hazard considered.

The term “game” mainly refers to an array of simulations that represent certain real-event situations, which are played and manipulated by the participants. The players are guided through the simulations to experiment with the outcomes of their decisions and learn from their playing experience (Dieleman and Huisingh, 2006; Huyakorn et al., 2012). A central function in TRPGs is the games master or moderator (GM), who leads the storytelling during the game simulations (Heinsoo et al., 2008). The GM acts as organizer, arbitrator and moderator of the multiplayer role-playing game and, therefore, they should know the game in detail and be able to answer questions regarding its rules. The games master's responsibilities could be split among two or three people (e.g. designers or organizers of the experiment). The basic role of the GMs is the same in almost all traditional role-playing games, although differing rule sets make the specific duties of the games master unique to that system (Tychsen et al., 2007). For example, in the ANYCaRE game the GM is responsible, among others, for providing feedback to the players about the hydro-meteorological observations, highlighting the relevant safety decisions that should have been taken at each playing round (similar to a weather reporter).

The roles to be played and the potential decisions and actions to be chosen by the players in ANYCaRE are predefined based on qualitative evidence gathered during European workshops that took place in March and April 2017 (Müller et al., 2017) and in previous research (Ruin, 2007). In particular, the game was designed in order to be adapted or easily adaptable to most European countries' warning and emergency decision-making contexts. Based on examples of warning systems from Switzerland (Canton of Bern), Spain (Catalonia), Italy (Liguria), Finland (South Savo) and France, commonalities of these systems were identified and used to simulate realistically the dynamics of the warning and response processes starting with the detection of a potential weather-related threat and ending with decisions related to the coordination of the emergency response (Fig. 1). This process involves the identification of three major components: (i) the type of actors involved in the warning system and their role in the decision-making process, (ii) the timeline/temporality of the warning phases, (iii) the types of actions and decisions the actors need to take in each phase to ensure people's safety.

Depending on the country and its administrative and legal organization, different types of actors may be involved in the warning system. The responsibility for emergency management is also dependent on the spatial scale of the event. Most often, the municipality is the first level of responsibility and the most important actor of the emergency management as soon as the event does not exceed the capacity or the boundary of this local level. Mayors are in charge of the activation of municipal emergency plans. Nevertheless, regional and national levels support their decisions with weather-related hazard forecasting and warning capabilities (meteorological, environmental or hazard-related expert institutes) and regional emergency management centres and rescue services. In times of emergency, actors with complementary competencies are gathered (at the administrative level of concern: local, regional or national in the case of a state emergency), either physically or remotely, to take decisions on how to best manage the crisis to ensure people's safety. Generally, those emergency operation centres (EOCs) include representatives of civil services as weather/hazard experts, police, firefighters and rescue forces, representatives of municipalities, and infrastructure experts from public or private companies (road, telecommunication, energy suppliers). A representative of the highest authority concerned will act as the leader of the group to organize the discussion and finalize the emergency decisions. The centre functions as the kernel of information, by receiving, checking and sharing information with operational teams as well as deciding upon complex emergency actions that need an holistic view of the situation, coordinating efforts and communicating with the public.

To reflect this standard type of crisis management organization, the ANYCaRE game proposes that decisions should be taken in the context of a simulated EOC gathering in the same room or a choice of the actors cited above. A panel of role descriptions that are distributed among the players (randomly or based on their real-life expertise) describes the tasks and responsibilities that each player has to contribute to the collective decision finalized by the group leader. Some roles, such as the ones of the forecasters or the one of the group leader, may be distributed carefully as they require either strong expertise or leadership. It is clear that the way those roles are played may influence a lot the results of the decision-making process and confidence of the overall group in their collective decisions. Nevertheless, random distribution of the roles may also raise participants' awareness about the capabilities and level of expertise necessary to interpret and act upon such complex, dynamic and uncertain crisis situations.

As shown in Fig. 1, the timeline or temporality of the warning process often relates to three phases (Müller et al., 2017). The first one is a detection phase in which model-based forecasts indicate the first signs of a potential high-impact event. At this early stage (3 to 10 days before the event), forecasts are neither accurate in space nor in time and the potential for impacts is difficult to assess. The second phase starts when the event becomes more probable and the location and timing grow less uncertain. This phase, starting about 2 days before the event, relies on model-based forecasts that are sometimes combined with some first empirical observations. At this stage, the first precautionary measures can be envisaged in order to prepare for the alert phase. In the third phase, the alert one, when the magnitude of the event is almost certain as well as its location and timing, it is time for emergency action. Decisions are mostly based on ground-truth observations made by different operational services involved in the emergency response. The temporality of those phases varies according to the type of hazardous events, as some are less predictable than others, like in the case of flash flooding vs. flooding events. Short-fuse events originating from convective storms, for instance, cannot be detected several days in advance, which contracts the warning and response process in a very short period of time. Based on these observations, the ANYCaRE game addresses this temporal aspect of the decision-making process by proposing a timeline adapted to the pace of the relevant hazard.

The experiment progression can be summarized in four progressive steps: (i) introduction to ANYCaRE, (ii) role assignment, (iii) simulations, and (iv) debriefing as proposed in Table 2. The experiment begins with the description of the setting by the GMs. Then, each player is provided with a certain role defining their responsibilities during the game. Players get a few minutes to become familiar with their role and to introduce it to the rest of the group before the main game simulations start. Following the principles of tabletop role playing, the GM facilitates the playing by presenting key aspects of the story and by triggering relevant discussion among the group of players. The designers' team act as observers of the playing process and based on their observations they facilitate the post-experiment debriefing. Based on the authors' experience, they believe a playing group in ANYCaRE should be preferably composed of 10 to 12 players.

The simulation aims to put players in the situation of acting out the roles of the different emergency management representatives gathered in the context of an EOC. As a group they are requested (i) to evaluate the weather-related situation and potential threat based on weather forecast information and their own assessment of the level of exposure, potentially supported by impact-based and crowdsourced information, (ii) to select, from a pre-established list, protective actions (if any) and four communication options that can be taken to best inform the targeted public about those decisions (see Appendix A for the details of the options available). The role of the emergency management group is to keep the population safe and ensure smooth execution of everyday activities in the territory while managing a given budget. In the case of weather uncertainty, this is a challenging task including a set of dilemmas. Deciding to alert the population and pushing them to stop their daily activities to take protective measures in areas not hit by a hazard might create unhappiness and loss of confidence in public authorities. On the other hand, people are looking forward to recreational events such as big festivals; potential cancellation requires careful consideration to preserve people's wellbeing without risking their security. Every decision taken by the emergency management group has a consequence, either in terms of human safety or wellbeing and economic value. The objective of the group is to undertake emergency activities from a predefined decision-reporting list, avoiding actions that might prove to be unnecessary at the end of the game. The proposed list of actions depends on the storyline and the purpose of playing and can be easily adapted to different scenarios to be played by different audiences. Thus, different versions of worksheets are created to fit the game implementation (see Appendix A).

Three rounds of decision-making are played successively to simulate the progression of the hazard from its early detection to its landfall. By using multiple rounds, we allow the players to experience evolving hydro-meteorological facets and test different decision-support tools, which give more and more accurate information, as the event occurrence gets closer. The repetition of the decision-making process over several rounds also helps the players to get better at managing their roles and learn from practice. Nevertheless, based on the dynamic and predictability of the simulated event, the pace of succession of the hazard and/or risk information and decisions to be made in the game can vary to represent a few hours to a few days in real life.

Based on weather-related hazard information, the players simulating the EOC need to take decisions related to the three phases of the hazard progression described in Sect. 2.1.2. At the beginning of the simulation, only weather and/or hydrological model-based forecasts are available for the coming hours or days. The level of uncertainty is still high. Round after round, more precise information including impact-based information is provided to reflect the decrease in uncertainty and the potential imminence of the event occurrence.

With this information, specifically distributed to each role with respect to its own responsibilities, the players first need to interpret and share their specific knowledge before envisaging and deciding upon solutions to use against the potential threat they identified. Based on their collective evaluation of the situation, they have a certain time to choose between 3 types of decisions: (i) stay aware and monitor the situation, (ii) take actions in the context of a warning phase and activate the EOC to take precautionary measures, (iii) activate the emergency plan and proceed to specific safety measures. For each of these types of decisions, several more detailed options are proposed on the decision worksheet (see Appendix A) to help the group decide whether they would provide some generalized advice for safety to the public (e.g. “If inside, move to higher floors”, “Be prepared for electricity disruptions”) or if they would proceed to more detailed emergency orders in specific area(s) of the territory (e.g. “Evacuate immediately”). Collective decisions are recorded by filling up one worksheet for the whole group in each game round. Each round is composed of two trials, one in which only existing basic hydro-meteorological forecasts are available and the second in which additional, more sophisticated decision-support products are provided. The group reports its choices in the relevant trial column on the worksheet at the end of each trial. Since certainty in the warning-response processes is a key element of efficient crisis management, the group is also asked to evaluate the level of confidence associated with their decision.

To reflect the time pressure that real-life EOCs always face, the players are given a limited time in which to provide responses to each trial and the GM is in charge of pressing the group to obtain their decision in time. For example, in the case of three-round simulations, each trial lasts 10 min in the first two rounds, whereas in the last round each trial lasts 8 min. Therefore, the first two rounds last 20 min each and the third round lasts 16 min (or less if the players are fast to take decisions on a given round). This time includes 1–2 min for the GM to present a short summary of the (hydro-) meteorological situation and consequences of the decisions that have been taken in the previous round. The experiment should preferably last less than 2 h (including the debriefing time). A detailed penalty scheme with costs in terms of safety, emergency resources, wellbeing, etc. may be assigned to each activity listed in the worksheet or not. If the objective of the playing is the training and or information/education then specific penalties may be not necessary. Triggering debate and discussion to enhance understanding is a major priority. If penalties are assigned, the designer(s) should ensure that every action has a straight consequence or penalty and that the initial credits given to the players are adequate, even for the worst-case scenario.

Inspired by European locations we introduce Anywhere City, an imaginary agglomeration including three distinct areas: A–C, located on the slopes and at the foot of highlands drained by two fast-reaction rivers (Fig. 2a). Area A is characterized by relatively steep tree-covered slopes drained by a small basin (e.g. 265 km2) and is known for its fast response to precipitation. A suburb of about 1000 residents and one school are located on the slopes. There are no permanent settlements or critical infrastructure in the flood-prone zone but one camping ground is located in the forest close to the riverbed (within the 10-year return period flood-prone zone). Area B is composed of both highlands and lowlands drained by a river basin of about 3000 km2. The densely populated urban area (e.g. 100 000 citizens) is located in the lower part of the basin. It includes the majority of schools, hospitals and other public services. About 30 % of the residential, commercial areas and public services are located in the 20-year return period flood-prone zone. Finally, area C is typical lowland with a large floodplain located in the lower part of a larger river basin (up to 4000 km2). There are no permanent settlements in C but the area surrounds the main bridge of Anywhere City, calibrated to resist a 50-year flood. The area is characterized by seasonal agricultural activity and a recreation place where the annual festival of Anywhere City, named “AnyDay”, is taking place.

The game takes place at the beginning of autumn and starts on a Monday, 5 days before the AnyDay festival takes place, with outside activities across the river and a big concert with famous singers close to the bridge area (area C in Fig. 2a). The peak of the festival is planned for Saturday, when participants are expected to reach numbers of 10 000. Public officials are checking out the weather forecasts to ensure that Anywhere City's traditional festival can happen under the safest conditions so that participants can enjoy the next weekend camping and celebrating in the region.

Each player is given a specific sub-role to act as representative of one of the following institutions: (i) hydro-meteorological services, which interpret the hazard model outputs and communicate warnings if needed; (ii) first responder services, which deal with the possible evacuation of residences, schools, campsites and public events; (iii) the municipality, which makes decisions related to the everyday (e.g. anticipation of school pick-up time, cancellation of school-related transport) or recreational events (i.e. AnyDay festival) in the city; (iv) road services, which manage road closures and the maintenance of the main bridge road in case of a flood emergency (see worksheet for flood scenario in Appendix A). The greatest challenge is to decide whether the AnyDay festival should be cancelled or if the event could be maintained and probably set up flood protection measures in the bridge area. Withdrawing such a big event, for which people have prepared and have been looking forward to, obviously would cause upheaval and would reduce their wellbeing. In this case, the municipality would also pay cancellation fees and other expenses with the ultimate goal to prevent people from risk. On the other hand, if high water or debris flow blocks and collapses the bridge while the festival is still on, thousands of unprepared people will be exposed to severe flooding. In general, if no protective action or evacuation were decided for the day an event hit an area, the population is considered subject to risk.

The period of concern in flood simulations are the 5 days of the week (from Monday to Friday) preceding the day of “AnyDay festival”. The GM provides and comments on medium-range deterministic precipitation forecasts and hydrological forecasts produced by the European Centre for Medium-Range Weather Forecasts (ECMWF) for Monday and Tuesday in order to familiarize the players with the products and slowly put them in context. Each of the 3 following days represents one round of the game for which collective decisions are requested from the players, assuming that every evening the group decides on the emergency activities of the next day (Fig. 3a). In each round, the players receive area-specific information so that they can make distinct safety choices adapted to the predicted hazard in each area of Anywhere City. Low-predictability events such as flash flooding on steep slopes and domino effects like debris flow in the river are also part of the flood scenario that prompts emergent stressful situations.

In the first trial of each round, players receive the 24-hourly accumulated precipitation based on the ECMWF ensemble mean for Anywhere City. Precipitation is calculated as the median value in every grid cell from 51 integrations with approximately 32 km spatial resolution. Secondly, input data include hydrological forecasts for three different locations (i.e. A–C) based on the European Flood Awareness System (EFAS), which is operationally running on the European scale under the Copernicus Emergency Management Services (Copernicus EMS) (Smith et al., 2016; Thielen et al., 2008). EFAS products were particularly selected because they present successful implementation of probabilistic medium-range flood forecasts with special attention on the communication of uncertainty (Demeritt et al., 2013; Pappenberger et al., 2013) and their EU-wide operational availability (Smith et al., 2016).

The second trial proposes additional radar-based high-resolution rainfall accumulation forecasts (i.e. every 15 min with lead times up to 6 h) produced by the European Rainfall-InduCed Hazard Assessment (ERICHA), and EFAS bias-corrected forecast hydrographs where near-real-time discharge observations are used to compensate the propagation of the error between forecasted and observed discharge (Bogner and Kalas, 2008). Advanced hydro-meteorological products are selected to show the importance of end-user tailored forecast visualizations and highlight the potentiality of combining (i) continental-scale medium-range forecast at coarse spatio-temporal resolution with (ii) local, high-resolution and short-range predictions to improve timely detection and better localization of extreme events. These include return-level hydrographs (i.e. time series of the return levels of corresponding forecasted discharge values at the same time step) and exceedance plots (i.e. daily box plots corresponding to different lead times of the hydrological forecast) from EFAS forecasts driven by the ECMWF ensemble.

The second group of products presented in the second trial focuses on the importance of translating the hazard into risk information, and especially probabilistic risk assessments, for the decision-making. Impact-based products are based on the Rapid Impact Assessment layer that combines event-based hazard maps with exposure information to assess several categories of impacts such as affected population and damage (Dottori et al., 2017) (more information available at http://anywhere-h2020.eu/catalogue/?product=river-flood-impact-forecast, last access: 11 March 2019). Direct economic losses are computed by combining the “Coordination of information on the environment” (Corine) map with flood hazard variables (i.e. flood extent and depths) and a set of damage functions derived for European countries. In addition to that, the extent of urban and agricultural areas affected is computed using the CORINE Land Cover inventory (Dottori et al., 2017). The last information presented to players emphasizes the benefits of the seamless integration of the volunteer geographic information (VGI) collected during the crisis from social media and via dedicated crowdsourcing applications (i.e. artificial tweets and posts in a hypothetical crowdsourcing system drawn especially for the experiment). VGI content is widely recognized as a crucial source of information that cannot be measured directly (e.g. socio-economic impacts, human perception), and thus, it is assumed to positively contribute to the overall situational awareness and crisis decision-making.

The strong-wind scenario was built based on the structure of the flood scenario with necessary adjustments. The experiment was originally designed for a combined training day of Finish civil protection services, meteorologists and electricity company employees in the South Savo municipality. Therefore, the territory and the game roles were selected as representative of the study area. The South Savo municipality has a special landscape with (i) dense forest areas, (ii) power lines above the ground that are vulnerable to strong winds and falling trees, (iii) lake areas attracting numerous boatmen during the summer, and (iv) the urban areas of Mikkeli and Juva. In the storyline, we illustrate areas A–C to represent the variety of these different areas (Fig. 2b). Area A represents the urban area of Mikkeli with a year-round population of 50 000. The game takes place in mid-July and starts on Friday, 24 h prior the Jurassic Rock festival and Sulkavan Soutu boat race. The music festival is expected to attract an additional 25 000 persons to Mikkeli (Area A) and 4000 boatmen to the boat race on the lake at Sulkava (Area B). The urban area also includes a lot of critical infrastructure like chemical factories, hospitals and shops that are dependent on a continuous electricity supply. Finally, Area C mainly contents the road leading from Juva to Sulkava, which is a crucial evacuation route for civil protection services when facing emergencies at the lake areas. Since the road tends to get blocked by falling trees in storms, it was included into the game storyline to create an additional challenge for the civil protection service's decisions.

The players of the strong-wind scenario act as (i) meteorologists, who interpret the NWP model information to the customers and issue warnings (and the corresponding meteorological bulletin); (ii) civil protection services, which make decisions for possible evacuation of public events (e.g. Jurassic Rock festival); (iii) electricity companies, which manage the maintenance of electricity distribution to the customers considering the related economic constraints (see worksheet for strong-wind scenario in Appendix A). The electricity company representatives are especially challenged to divide their resources effectively in order to fix potential power cuts as quickly as possible and ensure the electricity supply in the area. Similarly to the flood scenario, the biggest challenge for the players refers to the protection of the festival planned for Saturday.

The timeline in the strong-wind simulations is 24 h before the public events. Given the rapid development tendency, the small size and the short life cycle of convective storms, the forecast time is among the shortest lead times in weather phenomena, making forecasting a real challenge. Therefore, each round is chosen to be in terms of hours prior to the event (i.e. round 1 – 24 h, round 2 – 6 h, round 3 – 1 h ahead) in the ANYCaRE game (Fig. 3b). From meteorological perspective, the scenario imitates the so-called “Asta” strong-wind event that happened in 2010 and caused financial losses of over EUR 20 million (Astola et al., 2014). This particular event was selected because most of the trainees for whom the experiment was designed had indeed experienced strong Asta winds. Living through the event again (but without knowing it at the beginning of the game) and facing it under different circumstances, the players were offered the possibility to compare the existing tools to the new impact tools and to realize the possible consequences of such a strong-wind event in different situations (e.g. occurrence during daytime on the busiest summer weekend compared to the original event that took place during night).

The simulations replicate the KRIVAT briefing. KRIVAT is a platform supporting collaboration between enterprises managing critical infrastructure and authorities managing major disruptions to services (more information available at https://www.erillisverkot.fi/en/services, last access: 11 March 2019). During KRIVAT briefings different actors held a teleconference on Fridays to receive a weather outlook for the weekend. In the first round (24 h prior to the event), the GM provides information about the synoptic situation and the meteorological forecast of ECMWF and the Global Forecast System (GFS) in the area. The scenario also presents imaginary but realistic cascading effects and unpredictable events such as falling trees over the power lines and, subsequently, power outages in critical areas (e.g. close to the chemical factory). In such urgent situations, the civil protection service needs to react fast to cooperate and communicate with the meteorologists and the electricity company's staff. In each round, existing tools with only weather information are presented in the first trial, whereas the second trial provides additional impact-based tools. In particular, in the second trial, meteorologists receive model calculations outputs from the “Expert System for Consequence Analysis and Preparing for Emergencies” (ESCAPE) model calculations and interpret them for civil protection services. The ESCAPE model has been designed for evaluating the dispersion of toxic gases into the atmosphere and the effects on humans and the environment (Kukkonen et al., 2017). Second-trial tools emphasize the identification of the severity of the convective cells, their detailed route and the economic impact estimation that is particularly interesting to the electricity company. A common approach for the automatic identification and short-term forecasting of thunderstorms is the object-oriented storm tracking, which follows the movement of individual storms from remote-sensing data and extrapolates the storms based on the tracking information (Rossi et al., 2013). Severity categories of convective cells are classified by using failure data from the electricity company and comprise interesting input for forecasters and civil protection services. This information is assumed to also facilitate the optimization of electricity grid actions undertaken by the electricity company. Also, in the second trial, the civil protection service has access to the A4FINN platform and the available impact map. A4FINN, a product developed for the municipality of South Savo by the ANYWHERE H2020 project, combines weather model outputs and data from different sources with impact estimations. For instance, the emergency response plans of the civil protection services are now connected with weather information, which brings the experienced and less experienced emergency responders to the same level when making decisions. The platform provides a tool indicating the weather situation severity level in different municipalities. A4FINN includes relevant weather parameters and a map product, which shows the automatic, suggested preparedness level of the civil protection services.

It is important for participants to feel that the game experience was playable, the rules were understandable and the learning procedure was attractive (Dieleman and Huisingh, 2006; Turkay and Adinolf, 2012). After playing the ANYCaRE game, the majority of participants in the four conducted experiments expressed their satisfaction for the gaming experience. Based on the categorization of the comments provided on post-it notes during the debriefing sessions, we found that players noted their appreciation for the “stimulating” (six instances), “very fun” (four instances) learning approach experienced in ANYCaRE (see Table 4 for details of all comments and related instances). It was mentioned that it was “very interesting to try roles different from the usual day-by-day experience” (Helsinki player). The experiments were found to fulfil the main hypothesis, that the game offers a safe environment in which to enact different personas and gain a deeper understanding of the decision-making challenges met by the involved actors (eight instances): “The game is a good way to start handling emergency management with a different approach. Also, people that are not in their own role can understand the struggles of the others” (Grenoble player); “New experience that helps to understand better other roles” (Mikkeli player); “Good way to put you in the shoes of actors” (UGA player).

In all the experiments, the players recognized ANYCaRE scenarios as very realistic and presented a strong commitment to the storytelling (six instances): “Very real feeling and I really took the game seriously” (Grenoble player); “Very good game and incredibly realistic!” (Mikkeli player). During the feedback sessions in the Helsinki and Grenoble experiments as well as in the Mikkeli training, ANYCaRE players discussed that role playing was a good approach to bringing people from different institutions together to share knowledge and experience while examining new (pre-) operational tools and their potentialities. They often expressed their preference on ANYCaRE's role-playing structure for training decision-making skills on emergency management (four instances): “Would use this exercise in our later trainings for civil protection” (Mikkeli player). On the other hand, UGA students appreciated the game for educating them on their future responsibilities: “Good exercise/learning for the following because it gives concrete example”.

At the beginning of each round, the GMs described the existing weather-related conditions and explained if and why the group decision taken in the previous round was relevant or not to ensure safety with the minimum losses. It appeared that the players in the test experiments as well as in Mikkeli training already did a relatively good job in Trial 1 in the first round (e.g. by evacuating scouts from the campsite before the occurrence of a severe flash flood in area A in the flood scenario). Some players in the Grenoble experiment mentioned that improved forecasts including probabilistic information for exceeding return levels were very informative (four instances). Though, in some experiments it was observed that advanced meteorological forecasts such as high-resolution precipitation maps and flood probabilistic forecasts provided in Trial 2 did not necessarily lead to better decisions for emergency response. Still, this result may be subject to specific skills of the players who acted as forecasters as well as the overall experience of the participants in these sessions. For instance, operational meteorologists participating in Mikkeli training were able to detect significant signs about a potentially severe strong-wind situation already from the first round, even with a limited amount of information. Thus, they gave a very good briefing to the others, creating a confident atmosphere in which other actors could select emergency actions. This was not the case when the much less experienced UGA students played within the flood simulations. Although differences between Trial 1 and Trial 2 were not always obvious in terms of the selected emergency activity on the worksheets, in most of the cases, players in the four experiments rated their confidence more highly when they passed on to Trial 2.

In the debriefing, some players of the flood scenario simulations mentioned that detailed meteorological data are not trivial to non-forecasting-specialists (four instances): “Difficult information for operators in emergency centres” (Helsinki player); “For a non-expert, it is difficult to understand the probabilistic info provided by forecasting systems” (Grenoble player). Yet, hydro-meteorological forecasting was observed to dominate the decision-making and related discussion compared to impact estimations in the first two experiments. The limited use of the new impact-based tools in the game may be attributed to the lack of players' familiarity with the new products opposed to previously seen operational tools. Other potential explanations that require further examination may include (i) inadequate understanding on how to handle the new impact information during crisis, (ii) absence of trust in the new developments and (iii) incapability of the adopted visualizations to convey helpful information. In UGA experiment, where students were much less experienced with the provided hydro-meteorological products, decisions between trials 1 and 2 changed greatly considering the impact-based vulnerability information and, especially, the social media posts.

In the debriefing discussions, the players agreed that the information provided, in particular from impact estimations and social media, was very useful for better targeting their actions. Generally, in all the experiments players seemed to largely rely on impact observations assumed as reported through comments and pictures on social media. This was particularly the case in the third and last rounds of the three flood-scenario experiments in which the players completely changed their emergency decision after receiving a crowdsourced image showing the bridge blockage with wood and debris. The payers shifted from “no action” to the set-up of flood protection measures in area C, and finally, the closure and maintenance of the bridge road in Trial 2. The examples of modern impact-based vulnerability information included in the experiments were found to reduce the overall uncertainty in the decision-making process. The multi-model outputs were characterized as useful, especially when accessible to multiple actors directly without the need from mediation from other (governmental) agencies (three instances): “The ability to offer and share valuable data or information directly to consumer is brilliant. It avoids putting someone's safety in the hands of government decision makers” (Grenoble player). Expert forecasters and members of the civil protection service in Mikkeli experiment found that the variety of information presented through the A4FINN tool and impact-based maps helped them to shape a holistic view of the situation and increased their confidence in decision-making. The players of all experiments acknowledged that the variety of new products enhanced their sureness about choosing specific emergency activities and communicated in particular areas.