Communicating public avalanche warnings – what works?

Like many other mountainous countries, Norway has experienced a rapid increase in both recreational winter activities and fatalities in avalanche terrain during the past few decades: during the decade 2008–2017, 64 recreational avalanche fatalities were recorded in Norway. This is a 106 % increase from that of the previous decade. In 2013, Norway therefore launched the National Avalanche Warning Service (NAWS), which provides avalanche warnings to transport and preparedness authorities and to the public. Previous studies suggest that avalanche warnings are used extensively in trip and preparedness planning and have a relatively strong influence on the decisions people make in order to reduce risk. However, no evaluation concerning how efficiently the warnings are communicated and understood has been done to date in Norway. Avalanche warnings communicate complex natural phenomena with a variable complexity and level of uncertainty about both the future and the present. In order to manage avalanche risk successfully, it is fundamental that the warning message can be understood and translated into practice by a wide range of different user groups. Users with little or no avalanche competence may need simple information to decide when to stay away from avalanche terrain, while professional users may need advanced technical details in order to make their decisions. To evaluate how different modes of communication are understood, and how efficiently the informational content is communicated, we designed and implemented a web-based user survey. The modes of presentation were based on the Varsom.no 2017 version (Varsom.no being the national portal for natural hazard warnings in Norway). We first used a panel of 110 experts from NAWS to answer the survey, and used their answers to establish the indented message of the avalanche warning. We thereafter received responses from 264 users and compared their answers to those of the NAWS experts for the different modes of communication. We developed a method, the comprehension effectiveness score, to test the comprehension. Our empirical analyses suggest that most users find the warning service to be useful and well suited to their needs. However, the effectiveness of a warnings seems to be influenced by the competency of the user and the complexity of the scenarios. We discuss the findings and make recommendations on how to improve communication of avalanche warnings.

1 Introduction

Does the Norwegian Avalanche Warning Service effectively communicate its intended message? Risk communicators should pursue their intention to assess whether the message they disseminate is appropriate, understandable and useful (Charrière and Bogaard, 2016). This is a matter of prime concern during a period of dramatic change in information technology and information consumption in society. The Internet is rapidly becoming the main source of information, and studies show that communication is important: e.g. Brigo et al. (2016) conclude that Internet campaigns with emotional content are important to effectively promote awareness programmes on risk of avalanches and increase public knowledge related to these persistent and serious threats. This study focuses less on campaigns and more on the avalanche warnings and forecasts published daily by Avalanche Warnings Services (AWS^′).

In this paper we use the terms danger and hazard interchangeably. The same applies to the terms forecast, warning and bulletin.

1.1 Public avalanche warnings

In order to prevent avalanche accidents, AWS^′ throughout the world publish avalanche warnings to the preparedness authorities and the public. The standards for publishing danger levels and structuring the information in the warnings have been developed over the years with the aim of providing the users with a product that is as effective as possible. Most AWS^′ use the systems devised by the European Avalanche Warning Services (http://www.avalanches.org, last access: 9 September 2018; EAWS, 2017a, Müller et al., 2016) or the North American AWS^′ (https://avalanche.org/, last access: 9 September 2018; Statham et al., 2010, 2018). All AWS^′ quantify the danger into five levels (1–5) and use one or more of these standard elements: (1) a main (flash) message, (2) most avalanche-prone terrain (elevation; aspects), (3) avalanche problems, (4) snow cover and avalanche history, and (5) avalanche danger assessment and prognosis. The products from the different AWS^′ vary considerably in degree of detail, use of text, symbols and graphics, degree of advice provided, etc. (Burkeljca, 2013a). However, most avalanche warnings are typically structured in a standard journalistic inverted pyramid approach (Scanlan, 2000; Burkeljca, 2013b), where the most important information is presented at the top. More detailed and advanced information is sequentially presented further down in the pyramid. Accordingly, the standard EAWS approach (see https://avalanche.org/, last access: 9 September 2018) presents the danger level at the top, often accompanied by a flash message (a short main message). Secondly, the core zones (the most avalanche-prone terrain) are pointed out, typically by describing which elevation intervals and compass directions (sectors) have the highest danger. At this level, or at the level below, the current avalanche problems are described, followed by a description of the avalanche danger, snow cover and avalanche history, weather history and prognosis, and finally observations from the field. The pyramid approach also reflects what is useful to users at three different levels of competence (Mitterer et al., 2014): the top level of information targets all users, especially beginners with limited ability to understand and use complex information and users who want to get the key information quickly, the medium-level targets users with an intermediate to advanced knowledge of avalanche and snow assessments, while the detailed bottom level of information in mainly useful to experts.

The danger level ranges from 1 – low – to 5 – very high (termed extreme in North America) – and is an expression of the probability and size of expected avalanches in a given geographical region over a given period of time. In order to derive a danger level, the geographical extent should be above 100 km² (EAWS, 2017a). It is a generalization over a larger area, which typically has significant local variability (Jamieson et al., 2008; Schweizer et al., 2008; Techel et al., 2016). The European Avalanche Danger Scale (EADS; EAWS, 2016) was introduced in 1993 (SLF, 2018) and is used by all European AWS^′ but the Swedish AWS. The avalanche warning is a prognosis of expected danger over time, typically a period of 24 h, and is based on an analysis of the current snow cover and the effects of the weather on the snow and avalanche conditions during the prognosis period. The avalanche problems (Atkins, 2004; Landrø et al., 2013; Statham et al., 2018) describe the characteristics of the avalanche danger in more detail: the type of avalanche (dry or wet, slab or loose), trigger and failure mechanism, expected terrain locations, predictability and ease of detection. The level of detail varies between AWS^′, as do the number of categories. Advice for back-country travellers or preparedness authorities is provided by some AWS^′, either in the flash message (what to be aware of or do), as part of the avalanche problem (specific advice; is the problem manageable, and if so, how) or linked to the avalanche danger level (general advice). The snow cover and avalanche analysis provides a description of the snow properties and distribution relevant to avalanche conditions (e.g. snow height, recent snow fall, surface, wet/dry, critical layers) as well as recently observed avalanches in the region (e.g. locations, numbers, sizes, types, failure planes). The avalanche danger assessment provides further details on the avalanche threats, the distribution within the region, effects of expected weather, uncertainties, etc.

Introducing the EADS in 1993 as a European standard (Meister, 1995) improved communication of avalanche danger, and provided a basis for rule-based management strategies. The danger level is used by many users (Winkler and Techel, 2014; LWD Steiermark, 2015; Procter et al., 2014) and affects decision making during back-country tours (Techel et al., 2015; Furman et al., 2010) and in the work of risk management authorities. Avalanche warnings provide important information for back-country tour planning as well as en route (Winkler and Techel, 2014; LWD Steiermark, 2015; Baker and McGee, 2016).

1.2 Warning and risk communication

The purpose of warnings is to inform people at risk about the hazard and to promote “correct” and safe behaviour (Wogalter et al., 1999). To do so, warnings may assess not only threat and danger, but also exposure and vulnerability (WMO, 2015). Such impact-based warnings have been shown to be more effective than other types of warnings and are more and more in demand (DeJoy, 1999). Impact-based warnings facilitate informed decision making, which in turn leads to desirable outcomes and prevents unnecessary costs to society (Pielke Jr. and Carbone, 2002). In the case of avalanche warnings they provide users with both general and specific information about the current and expected levels of avalanche danger, the type of avalanche problem at hand, and behavioural advice. The main aim of the warning message is to inform the user about the nature and severity of current and expected threats, and about how he or she can mitigate the risk or avoid the threats. However, since most regional AWS^′ do not provide specific and local descriptions of the forecasted risk, it may be difficult to effectively reach this goal. In addition, most AWS^′ lack detailed information on the type of objects and number of individuals who are at risk, and on the exposure and vulnerability of these. Thus, most AWS^′ provide impact-based warnings in a general sense, but not in terms of impact specific to detailed geographical locations, people, roads, and so on. An AWS issues regional forecasts, which describe the general probability and size of avalanches in a geographical area. These forecasts may describe the general impact for recreational activity, roads, buildings, etc., but will not be able to determine the nature or scale of the impact on individuals or objects. For example, if an AWS^′ issues a warning at level 5 extreme danger in a mountain area where no one is recreating, the impact will be nil as nobody is exposed to the danger. At the other end, this warning describes the impact to people that are at risk, if people choose to travel in avalanche terrain in extreme danger. So in a way the forecast is impact-based, but it cannot quantify the impact specifically as the AWS^′ will not know how many people are exposing themselves to the risk. The warnings advise the users on how to reduce or avoid being exposed and vulnerable to the avalanche danger, and thus the risk.

Although risk communication research has been a growing field since the 1980s (Sivle, 2016), some researchers report that warning practices have not changed much during the past decades (Kasperson, 2014), and there is sometimes a gap between the intended message (warning) and the message received (Gigerenzer et al., 2007). A range of factors contributes to this gap. One such factor is that many people find it difficult to interpret numbers and probabilities. People's ability to make meaning out of numbers and statistics is often referred to as numeracy (Lipkus and Peters, 2009). Both large-scale surveys and small-scale experiments show that many individuals lack this ability (Låg et al., 2014; Kirsch et al., 2002) and that even well-educated individuals often display a low level of numeracy (Lipkus et al., 2001). One approach to mitigate this problem is to use qualitative explanations with words like “likely” and “unlikely”. Even though people vary in their understanding of such words, users can conceptualize the concepts by comparing them to risks they already understand (Gordon-Lubitz, 2003; Edwards and Elwyn, 2001).

Another reason for a mismatch between the intended and received messages is that people vary in their motivation to use, and competence to read, warnings. The level of use and understanding of the information provided in the warnings vary between different user groups, and between different geographical regions (Wogalter et al., 1997). Geographical differences are driven both by differences in the characteristics of the user groups present in the area and by differences in the complexity and amount of supporting information provided by the regional AWS^′s (Burkeljca, 2013a). Differences in the use of warnings may further be driven by variations in the level of trust in authorities and experts, and by personal experiences of natural hazards (Wachinger et al., 2013). Avalanche danger may in fact be so complex that a novice will not be able to manage the same terrain as experts, no matter how well the warning is communicated. The avalanche warnings are communicating a phenomenon that many users conceive as a low-probability event, since many users never or seldom experience a release of an avalanche themselves. This conception may in itself reduce engagement on the users' side and interest in reading and using the avalanche warning, and reduce interest in investing in understanding the warning. Another challenge is that the warnings are used in several different ways, which also could lower the interest.

Taken together, these differences make it difficult for providers of avalanche warnings to meet the needs of all groups. The challenge facing providers of avalanche forecasts is made even more difficult by the lack of research on how efficient different ways of presenting the avalanche danger to different groups are. For example, less competent and motivated users may need simplified explanations and direct travel advice in order to be able to use the information. They may easily be overloaded if the warning contains a lot of detailed information (Maltz, 2000; Liang et al., 2006). For advanced users, on the other hand, simplified information and advice may be of limited use. Instead this group may demand detailed information about the snow cover. It can be challenging to simultaneously satisfy the needs of both groups.

1.3 The Norwegian Avalanche Warnings Service, Varsom and RegObs

During the past few decades, Norway, as many other countries, has experienced a rapid increase in recreational winter activities in avalanche terrain (mainly ski touring, snowmobiling and to some extent snowshoeing). The increase in back-country recreation has unfortunately been associated with an increase in fatal avalanche accidents. During the decade 2008–2017, avalanches claimed 64 recreational fatalities (61 % occurred in northern Norway and Svalbard); the corresponding number for the decade 1998–2007 was 31 (NGI, 2018). By contrast, avalanche fatalities in houses and during transportation decreased from 7 in 1998–2007 to 2 in 2008–2017 (NGI, 2018). Similar trends are reported from other parts of the world (e.g. Techel et al., 2016). Another three fatalities, all recreational, were recorded during the 2018 winter season. Thus Norway had eight fatalities the last 3 years, which brings the annual fatality figures down by 50 % as compared to the previous decade.

In other states with significant increase in the use of avalanche terrain, such as in the US and several European countries, Avalanche Warnings Services (AWS^′) have succeeded in avoiding an increase in fatalities, although their warning styles and formats have varied quite a bit. The trend in the US has been a declining fatality rate: whereas the number of fatalities has been rather constant, the use of avalanche terrain has surged (Birkeland, 2016).

Table 1 A description of the elements included in the avalanche warning on Varsom.no. Figure 2 shows how the elements are shown in relation to each other on a smartphone web browser.

In order to halt the undesirable trend in avalanche accidents in Norway, the Norwegian Government in a white paper in 2012 decided to establish the Norwegian Avalanche Warnings Service (NAWS) in January 2013 (Engeset, 2013). NAWS publishes regional avalanche warnings for Norway, including Svalbard, on a daily basis on the web portal http://www.varsom.no (last access: 9 September 2018) (Johnsen, 2013). The Norwegian Water Resources and Energy Directorate owns and operates NAWS in collaboration with the Norwegian Public Roads Authorities and the Norwegian Meteorological Institute. The reduction in annual fatalities during the previous 3–4 years suggests that NAWS is effective, as the accident numbers have not increased although the use of avalanche terrain for ski touring has increased drastically in Norway, every year during the last decade or so.

In 2017, regional avalanche warnings were issued for 21 regions in Norway (Fig. 1). In addition, warnings were issued for the rest of the country when the avalanche danger was expected to reach danger level 4 or 5. An example of an avalanche warning on Varsom.no is shown in Fig. 2. The avalanche warning published on Varsom.no includes the elements described in Fig. 2 and Table 1.

Four elements (danger level and main message, region map, avalanche problems, and mountain weather prognosis) are available in an English version of the warning on Varsom.no, while two elements are in Norwegian only (avalanche danger assessment and snow cover history) and one is partly in English (RegObs-feed with observations).

Figure 1 Screen dump from Varsom.no showing the avalanche warning regions in Norway. Daily warnings are published for the regions coloured according to the danger level. The other regions (grey) are monitored and warnings are only published at danger levels 4 and 5.

Figure 2 An example of an avalanche warning as issued on Varsom.no in 2017. The numbers refer to the elements analysed in this study. (1) Danger level and main message, (2) avalanche danger assessment, (3) region map, (4) avalanche problems, (5) snow cover history, (6) mountain weather prognosis and (7) RegObs-feed with observations. The figure shows the screen dump from a smartphone, with the middle and right panels showing the screen as the user scrolls down the page. The warning has been translated from Norwegian to English.

All text but that of the main message and the avalanche problems is written manually by the NAWS forecasters. Sometimes forecasters use parts of the text from the previous day, especially in the snow cover history. The text in the main message is produced in the following manner: the forecasters get a list of text suggestions that are available depending on the chosen danger level and avalanche problem(s). He/she may choose to use text from this list and edit it, write the message from scratch or copy the text from the previous day. The text in the travelling advice in the avalanche problem is generated from a list of text suggestions. The selection of text is based on the combination of the chosen danger level and avalanche problem. The forecasters may edit the text afterwards. NAWS generates text suggestions in the forecast editing software for the main message and avalanche problems in order make the text in the warnings easier for the users to read, ensure that the terms and wordings are as good and consistent as possible, make sure the time needed to produce the text is not too high and make translation of the text to English as good and as easy as possible. NAWS has been having discussions about the degree to which text is to be created from scratch by individual forecasters or rather be pre-defined or suggested from a standard library of sentences and terms. NAWS uses a hybrid approach to this, and the creative from-scratch text is mostly found in the main message (Norwegian and English; the English version is sometimes an extended version of the Norwegian in order to incorporate more detailed information about the snow cover and avalanche danger), avalanche danger assessment (Norwegian only) and snowpack and avalanche history (Norwegian only). However, creative text may suffer from poor language and significant individual differences that are difficult for the users to understand. Also, much time may be spent writing text to convey a message that has already be written in a much better way by someone else. However, the interest and motivation of users may drop if they get the feeling that too much text is auto-generated or copied and pasted. NAWS is aware of this effect and continuously makes efforts to prevent this from happening.

Since the start in 2013, NAWS has continuously worked to improve both the competence level of observers and forecasters, and the system for presenting the forecast. User feedback suggested that most users find the warnings useful and of high quality. However, to date, no formal evaluation has been done of how effective NAWS is at communicating its intended message. Such an evaluation is important, as public avalanche warnings have only been available in Norway since 2013 and Norwegian users are less used to using the warnings to manage risk than users in countries with a longer history of public avalanche warnings.

In order to improve the avalanche knowledge in the Norwegian population in general, and the ski touring and snowmobiling communities in particular, NVE launched the “Snøskredskolen” (avalanche school) on Varsom.no. The avalanche school is a tailor-made resource for users of the avalanche warnings, as all key terms and concepts are explained and safe travelling advice is provided. It is also a much used resource for avalanche course providers.

As a system in the Varsom.no portfolio, RegObs provides data from the field as a basis for making forecasting decisions. RegObs is an open web- and app-based system for reporting, storing, querying and sharing observations and assessments from the field with the forecasters and the public. The observations are public and a live feed of observations is displayed on Varsom.no, next to the avalanche warning. As such, RegObs is an integral part of Varsom.no and the communication of the avalanche warnings. RegObs communicates the field observations and assessments that the warnings are based on, in a transparent way. As far as we are aware, RegObs is the only open-access online real-time distribution system for avalanche forecasting observations right now, although previous efforts have provided open access to accident data (Duclos et al., 2008). As far as the authors are aware, there is no other completely open-access online real-time distribution system. Other systems, such as the InfoEx, the Mountain Information Network and the Mountain Hub, lack open APIs or restrict access to functionality or data partly or fully.

1.4 Aims of the study

The current study is part of a larger project, which focuses on communication of flood, landslide, and avalanche danger warnings. In this study, we evaluate the efficiency of warnings by the NAWS on the website Varsom.no. Avalanche warning systems are used in trip and preparedness planning, and have been shown to have an influence on the decisions people make in order to reduce risk (e.g. Furman et al., 2010; Marengo et al., 2017). Mountain guides, course providers, rescue services and avalanche observers report that people actively respond to the avalanche warnings on Varsom.no, and to a large degree choose snow, terrain and time/day for travelling according to the danger level, avalanche problem and advice provided by NAWS.

Warnings should therefore ideally be revealing and unambiguous. To assess whether the warnings published by NAWS fulfil these requirements, we asked the following research questions: (1) which risk factors are considered most difficult to assess and manage? (2) Which elements in the warning are considered most and least important? (3) Which elements are easily misunderstood or considered poorly communicated? (4) What kind of information and features are missing or ignored by users?

We tested whether users interpreted the danger and behavioural implications differently depending on whether the message was described by text, by symbols or by pictures. Furthermore, we tested how well the warnings were understood, by testing four alternative ways of communicating two different danger scenarios. We developed a method, the comprehension effectiveness score, to test the comprehension.

2 Methods and data collection

We developed a web-based questionnaire and survey to collect data for the study. Questionnaires are useful tools for acquiring information on public knowledge and perception of natural hazards, and can provide valuable information to emergency management agencies for developing risk management procedures (Bird, 2009). This chapter presents the methods, participants and survey design of this study.

2.1 Participants

NAWS personnel participated in the expert survey (personal invitation only), while all types of users participated in the user survey (open invitation, anyone could participate). The user survey participants included all types of users (various degrees of competence and experience, from beginners/novices to experts; various types of use, from recreational to professional and preparedness), while the NAWS expert survey included only forecasters and observers, all trained in the same system.

2.1.1 NAWS expert survey

The aim of the first survey was to derive a set of “correct” answers to questions on the meaning of the presented warnings. We therefore invited 200 avalanche experts (mainly avalanche forecasters and observers in the NAWS) to participate in the survey during the period 15–26 October 2017; 110 experts provided complete responses. Of these, 67 were observers, 21 forecasters and 22 were not active in the forecasting. The last group consisted of former forecasters and observers, and of individuals with a professional liaison role in the forecasting services. Of those providing details about gender, 25 % of the participants were women and 75 % were men.

2.1.2 User survey

The purpose of the second survey was to test how well the NAWS message was understood by non-expert users, and therefore targeted users, and potential users, of the NAWS. We recruited participants via social media, Varsom.no and different user-related web pages.

A total of 485 respondents answered the user survey. Not all respondents answered questions in all sections, leaving 264 respondents for analysis of Section B (avalanche warning), 222 respondents for analysis of Section C (text versus symbols and pictures) and 177 respondents for analysis of Section D (comprehension). The lower number of user respondents completing Sections C and D reflects a common challenge in web-based surveys to engage participants enough to answer complex and time-consuming questions.

Of those proving details about gender, 17 % of the participants were women and 83 % were men. The mean age in the sample was 35 years (min = 19, max = 69); 26 % of the respondents lived in northern Norway, 8 % in Trøndelag, 11 % at the north-western coast, 24 % at the western coast, 27 % in the south-east, 1 % on Svalbard, and 2 % answered other.

2.2 Survey design

In order to obtain valid responses and avoid careless responding, it is important that participants are motivated to take the survey, understand all questions, feel that they can answer the questions, and do not lose interest before the end of the survey (Meade and Craig, 2012). We therefore pretested and revised all survey items in an iterative process. In the first stage, NAWS personnel, both forecasters and observers, provided qualitative feedback on how well the avalanche warnings communicated the message that NAWS would like to disseminate, and this was taken into account when the questions and response alternatives were designed. We thereafter asked a test panel consisting of project members (N=12) to provide iterative feedback on the content and structure of the survey. Based on the feedback from the NAWS personnel and the test panel, we rephrased several questions and instructions to improve clarity. We also reduced both the number of questions and response alternatives. The latter shortened the completion time of the survey to about 10 min. The survey was constructed so that it was possible to view and answer all questions using a variety of devices, including smartphones. The general structure and purpose of each section of the NAWS expert survey and user surveys are described below.

The five sections (A–E) were identical in the NAWS expert survey and the user survey. Sections B–D provided the core data for the analysis in this paper. An overview of the survey is provided in Table 2.

The two scenarios in Section D were based on accessing the warnings (in Norwegian) on Varsom.no at the time of the survey (autumn 2017). The four alternatives given for each of the two scenarios in Section D were picked randomly for each user respondent. We did not counterbalance the order: all respondents received first the level 2 scenario and then level 4, but with different alternatives for each scenario.

2.3 NAWS expert survey

As briefly mentioned above, the main purpose of the expert survey was to derive a template of “correct” answers. More specifically, we wanted to identify key information elements and define sets of behavioural implications in different avalanche danger scenarios. In other words, we used the experts to operationalize the intended content of the avalanche forecasts. To make sure that the operationalization was valid, we used a relatively large and heterogeneous group of avalanche experts.

To limit completion time and mental strain for participants, we only used two avalanche danger scenarios (level 2 and level 4; see section D in Table 2). Each expert was randomly exposed to one out of four alternative ways to present the forecast for each danger level (Fig. 3): (1) avalanche danger level with explanation, (2) avalanche problem with technical details, (3) avalanche problem with technical details only, and (4) avalanche problem with advice only.

After the expert respondent had read the example, we first asked him or her to rate how well the danger was communicated in the example, on a scale from 0 to 10. We thereafter asked the expert to identify key information elements and behavioural implications of the avalanche forecast. The options were pre-defined, as described in Table 3. We were specifically interested in identifying the most important message that the forecast aimed to communicate.

Figure 3 Alternatives 1 to 4 used for the two scenarios: (a) level 2 and wind slab (upper panel) and (b) level 4 and wet slabs (lower panel). The text has been translated from Norwegian into English.

Table 2 Overview of the survey.

2.4 Communication effectiveness score

In order to establish a communication effectiveness score, we used the NAWS expert answers to allocate weights to the different behavioural implications. We allocated a positive weight of +1 to elements positively identified as important by more than one out of three experts (33 %), and a weight of −1 to elements positively identified by less than one out of five experts (20 %). All other elements were given a weight of null.

Our reasoning behind using positive and negative weights to calculate the communication effectiveness score is that there is no objective correct answer. Accordingly, we use the NAWS expert answers, where a factor receives +1 if the majority of experts provide support, 0 for inconclusive support by the experts, and −1 if a small minority of experts regard this factor as relevant. The rationale behind this approach was to give a penalty (a weight of −1) to statements that were selected by few/no NAWS experts and a point to statements that were selected by many NAWS experts. In the design phase, we explored using different algorithms for calculating the scores, for example by using the relative number of experts selecting the statement as a weight or decimal weights. However, in order to keep the method and results relatively easy to understand and interpret, we choose a straightforward approach.

The expert choices and resulting weights are listed in Table 3. As can be seen in Table 3, many experts agreed on the most important implications, and very few items are therefore close to the cut-off value. Nevertheless, to ensure that our results do not hinge on our chosen levels (33 and 20 %), we have tested both upward and downward variations of the cut-off values. The results presented in Sect. 3 are robust to these variations.

2.5 User survey

The user survey was open to the public during the period 1 November–15 December 2017. We published links to the survey on a relatively wide set of platforms: Varsom.no, the free online skiing magazine friflyt.no, and on the Facebook page of the most popular weather service in Norway, YR.no. The association of snow scooter clubs (Skuterklubbenes fellesråd) and the Norwegian Hiking Association (DNT) kindly distributed the survey to their members. Finally, we announced the survey on the Nordic avalanche conference in Åndalsnes in the beginning of November.

Each participant was asked to answer the full survey (Sections A–E). In Section D, the users were, just like the experts, randomly exposed to one out of four alternative ways of presenting the avalanche warning for the level 2 and level 4 scenarios, and thereafter to first rank how well the danger was communicated on a scale from 1 to 10, and to mark the most important behavioural implications of the warning.

Table 3 Expert survey results (number of respondents selecting the statement, in %) and design weights established for a communication effectiveness score.

We used the weights in Table 3 to calculate a “communication effectiveness score” for each participant and each behavioural implication. To illustrate, consider a user respondent who ticked the boxes for statements 1–3 after reading an example of the level 2 scenario. Based on the scores in Table 3, we would give this user a score of −1 (the sum of $-\mathrm{1}+\mathrm{1}-\mathrm{1}$). If the user instead ticked the boxes for statements 3 and 5 after reading an example of the level 4 scenario, we would give him or her a score of +2 (the sum of $+\mathrm{1}+\mathrm{1}$). The scores for the level 2 scenario ranged from −4 to +4, and for the level 4 scenario from −6 to +3.

2.6 Web survey or field testing

Our overarching aim for this study was to investigate users' comprehension of the warning. Ultimately, all public warnings aim at making people take the correct actions at the correct time. However, there is a large body of evidence demonstrating that there is a mismatch between what people say and what they do (e.g. Jerolmack and Khan, 2014). Therefore, if we studied people's behaviour and not comprehension we would not know whether the lack of correct action was due to lack of comprehension or rather a mismatch between attitudes and behaviour.

Self-reports are by many accounts not a perfect method, but in this case we found them to be the best approach to test people's comprehension. In addition they allow us to collect a substantially larger number of respondents compared to for example a field study or interviews. A web-based survey is also relevant, as many decisions are made based on reading the avalanche warning on the web ahead of getting into navigation in the terrain. It could be decisions such as to choose forest rather than alpine for today's trip, or to delay the planned trip a few days until the snow stabilizes. However, it would be very interesting to test what people know and also what they do. This would call for a different study altogether, but is a very good idea for future research.

2.7 Ethics

This study registered anonymous information exclusively and did not collect data that can be used to identify individuals. All respondents actively gave their consent for the use of the data for research and the project.

3 Results

In this chapter, we present the avalanche-related demographics of the user respondents (sections A and E), well-functioning and malfunctioning parts of the 2017 version of the avalanche warnings on Varsom.no, as perceived by the participants (Section B), the participants' evaluation of how well text, symbols and pictures assist the informational content in the warnings (data from Section C), the participants' evaluations of how well different levels of complexity in the text persuade the informational content in the warnings (data from Section D), and test results for level of comprehension at different levels of complexity in the warning texts (also data from Section D).

3.1 Demographics

The statistics of the users with respect to competence, experience, activities and geography are listed below and in Table 4.

• 14 % of the user respondents had no or little avalanche knowledge (labelled “None” in Table 4), 27 % stated that they had avalanche-related competence but no formal training (“Competent, no course”), 48 % stated that they had avalanche-related competence and formal training (“Competent, course”), and 10 % were avalanche instructors or professionals (“Expert”).

• 82 % stated that they had used avalanche gear (e.g. avalanche beacon, shovel and probe) for several seasons, 7 % for one season only, and 11 % had never used this type of equipment.

• The majority of users stated that their main activity in avalanche terrain was alpine ski touring (66 %). Relatively many users also stated that they engaged in off-piste skiing (32 %), or Nordic mountain skiing (23 %), while relatively few said that they travel in avalanche terrain by foot (9 %), on a snowmobile (7 %), or on snow shoes (3 %). Three percent stated that they engage in other types of activities in avalanche terrain. Note that the users could chose multiple activities.

• Concerning the use of NAWS, 76 % of the users answered that they always use the avalanche warnings, 21 % that they use the warnings on a regular basis, and 3 % that they rarely read the forecast.

Many of the respondents have avalanche-related competence and formal training, and many use the avalanche warnings on a regular basis. This suggests that the sample of respondents could be biased towards a population with more avalanche expertise than average.

Table 4 Contingency table of user respondents' experience (number of tours in avalanche terrain per year) versus competence.

3.2 Avalanche warning

A total of 264 user respondents completed the questions in Section B. In this section, we asked the respondents to identify risk factors that they perceived difficult to manage or mitigate, parts of the avalanche warnings that they perceived difficult to understand, and important information perceived to be missing in the avalanche warnings. Key results from the 110 respondents in the NAWS expert survey are also presented in this chapter for comparison.

3.2.1 Avalanche risk factors considered difficult to assess and manage

In order to find out what the users consider to be most difficult to assess and manage, we asked “Which factors are most difficult to assess and manage in order to complete a safe trip?” The respondents could choose multiple factors. Available factors and results are shown in Fig. 4. The results show the following.

• The vast majority (87 %) of the users perceive the snow cover to be the single most difficult factor to assess and manage. This judgement does not depend on the respondent's experience or competence (χ² test, p=0.516 and p=0.403, respectively); 86 % of the NAWS experts considered this factor to be the most difficult factor.

• 34 % of the users perceive other people in the group to be the most problematic factor. More than every second NAWS expert (51 %) rated this as the most difficult one.

• Among the users, there is a relatively even distribution of individuals who perceive terrain traps (28 %) and weather (25 %) to constitute the other most problematic factors.

• Steepness is perceived as a problematic factor among relatively few respondents.

Figure 4 Factors users and NAWS experts considered difficult to assess and manage in order to have a safe trip in avalanche terrain.

3.2.2 Avalanche risk factors considered most and least important

In order to find out what the users consider to be the most important element in the warning, we asked “Which elements in the avalanche warning are most important?” The respondents could choose multiple answers. Alternatives and results are presented in Fig. 5. The results show that the users perceive a relatively wide range of elements in the warning to be important.

• A majority of the users state that the avalanche assessment (69 %), the avalanche problems (67 %) and the main message (65 %) constitute the three most important elements in the warning.

• About half of the users consider the snow cover history (56 %) and the danger level (48 %) to be important.

• Over a third of the users consider snow and avalanche observations (37 %), mountain weather (39  %) and management advice (42 %) to be important.

We find no evidence that the elements chosen as most important depended on age, gender or experience (linear regression R²=0.022, p=0.224). The NAWS experts rated the avalanche problems as the most important factor (77 %), followed by the avalanche assessment (62 %), the main message (57 %) and the snow cover history (55 %). The danger level was considered most important by 39 % only.

Figure 5 Factors the users and the NAWS experts considered most important in the avalanche warning on Varsom.no.

In order to find out what the users consider of least importance or use, we asked “Was anything of little use or importance? You may elaborate on the problem being format, content or other”. A total of 69 participants responded to this question. Twenty of these provided positive or neutral comments. We summarize the critical feedback, and our interpretation of this feedback, below.

• Seven users stated that they found the mountain weather to be superfluous, and that they rather used the standard weather forecast. Thus, clarification in the difference between the weather forecast and the summary of the mountain weather, and the link between the mountain weather history and forecast, and the avalanche forecast, is recommended.

• Five users stated that the warning contained too many, and complex details and information. These users were mainly novices. This may imply that users with less skills and interest in the topic fail to get the key messages.

• However, another set of six users considered the level of detail to be too low. These users stated that the usefulness of the warning would be higher if it were less general, and if the forecast region was smaller. These answers point to the possibility that general forecasts for relatively large regions reduce the attention paid to the warnings.

• Three users found the core zone sector diagram to be problematic. More specifically, these users found it difficult to know whether dark sectors represent safe or unsafe regions. Although only three users commented on this, their feedback is important since it implies that some users of NAWS may chose the unsafe sector because they misunderstand the graphics. See also Sect. 3.2.3 for related results.

• Finally, four users found the snow and avalanche observations sometimes to be too complicated or described in too difficult terms.

Figure 6 Test of what communicates the avalanche problem best in the avalanche warning: (a) text, (b) symbols or (c) pictures.

3.2.3 Elements easily misunderstood or poorly communicated

A total of 95 users provided comments on whether the avalanche warning contains parts that are easily misunderstood or poorly communicated. Thirty of the comments were positive or neutral. We summarize the critical feedback, and our interpretation of the comments, below.

• Eleven users found the core zone sector diagram to be easily misunderstood. Like in the case of users who stated that the core sector diagram to be of little use, these users stated that they found it difficult to know which of the sectors (dark or light) that are most dangerous. Some users suggested to add a legend or use more or different colours. These findings corroborate the findings in Sect. 3.2.2.

• Another 11 users perceived the regional warnings to provide too little detail in terms of spatial or temporal variability, and that the forecasted regions were too large. These findings corroborate the findings in Sect. 3.2.2.

• Eight users found it difficult to understand the danger level, in terms of the meaning and consequence of it for the user. This is important, because if users do not understand the meaning of the danger level, they are poorly equipped to manage their risk exposure.

• Finally, six users stated that the large amount of information provided in the warning made it difficult, especially for beginners, to decipher the key message. This corroborates the findings reported in Sect. 3.2.2, where five users stated that the warning contained too much detail and information.

The answers from the NAWS expert survey suggest that experts perceive similar factors to be as problematic as users do: i.e. the core sector and elevation diagrams, spatial and temporal variability, danger level, and uncertainty. However, the NAWS experts also pointed to a few problematic factors not mentioned by the users: avalanche size (especially the name “small” used for size 2), probability and distribution. Note that the EAWS is changing the denominations used for avalanche sizes during 2018, which will resolve the problem with communicating size 2 avalanches.

3.2.4 Missing information and features

In the final part of Section B, we asked the respondents to identify missing information in the avalanche warning. Sixty-seven respondents provided comments. About 20 of these stated that no important information was missing. The elements asked for by the remaining 47 participants were the following:

• observed weather and snow, and links to more detailed observations;

• ATES recommendations (Avalanche Terrain Exposure Scale is a method for classifying the degree of terrain avalanche-exposure, Statham et al., 2006);

• advice connected to competence levels, and

• more detailed warnings/information. Better visualization of important weak layers (depth, type, etc.).

We also asked the participants whether some information or features are missing in the RegObs application. Eighty-one users responded to this question, of which about 35 responded that they did not use the application or were indifferent. The users asked for the following to be included in future releases:

• weather data,

• a possibility to enter and record snow profiles,

• a possibility to read the avalanche warning (at least the danger level and avalanche problems) in the application,

• an opportunity to track trips,

• a more user-friendly interface,

• access to avalanches and avalanche paths,

• information about actual elevation in relation to the avalanche problem elevation range, and

• easy access to the snow cover history and relevant recent snow profiles nearby.

Several of these features are being implemented by the time of publication of this study.

The results from the NAWS expert survey suggested that these pieces of information in demand:

• more precise description of where in the region or terrain the avalanche problem is expected, and where the danger level is expected to be lower, and

• a better description of the uncertainty and local variability.

3.3 Testing of text versus symbols or pictures

In section C, we asked the respondents to rate how well text, icons, and pictures communicate the avalanche problem on a scale from 1 = poor, to 3 = good. Each respondent evaluated two types of avalanche problems: a wind slab, and a persistent slab (see Fig. 6). A total of 222 user respondents completed this section of the survey.

Figure 7 (a) User rating (left) and comprehension score (right) for the level 2 wind slab scenario; (b) user rating and comprehension score for the level 2 wet slab scenario.

Table 5 Results from the test of what communicates the avalanche problem best in the avalanche warning.

The results show that users preferred text and symbols to pictures (Table 5): 89 % rated the new EAWS symbols as good or OK. The users were familiar with the names of the avalanche problems, which have been presented as text on Varsom.no during the previous three seasons. The users were not familiar with the symbols, as they were introduced at Varsom.no for the 2017/2018 season after being introduced as an EAWS standard in June 2017 (EAWS, 2017b). Pictures have not been used in the warning on Varsom.no, but a few users may have seen the pictures in the avalanche school at Varsom.no. Notably, we found that the symbols were rated more positively the more experienced a respondent was: ${\mathit{\chi }}_{\mathrm{203}}^{\mathrm{2}}=\mathrm{15.26}$, p=0.018. The text and pictures were rated equally irrespective of one's experience: p=0.338 and p=0.543, respectively.

3.4 Testing of comprehension of the two scenarios

A total of 177 user respondents completed the test for comprehension in Section D by responding to one of the four alternatives for each of the two scenarios. To recap, we asked the respondents to (1) rate how well they perceived the avalanche danger to be communicated, (2) what the most important behavioural implications of the warning were, and (3) what advice they would give to others based on the warning message. We measured how well the danger was communicated in the warning on a scale from 1 to 10. Of those who provided answers to this question, 21 % gave a rating of 10, and 56 % a rating of 8 or higher. Only 14 % gave a rating of 4 or lower. Mean ratings for the two scenarios (danger level 2 and level 4) and for each of the four alternatives are presented in Fig. 7 below (left column). Figure 7 also depicts the comprehension scores (right column). Higher scores indicate a higher match between the behavioural implications chosen by the users and the NAWS experts. For the danger level 2 scenario the minimum score is −4 and the maximum score is +4, while scores for the danger level 4 scenario range from −6 to +3.

We next compared the user ranking and comprehension score in more detail, by (a) comparing the comprehension score to a score of 0 and (b) investigating whether user ranking or comprehension differs between the four alternatives with ANCOVAs where a user's experience was a covariate. For statistical analysis we used JASP (2018).

3.4.1 Danger level 2 wind slab scenario

For the danger level 2 wind slab scenario, the average user ranking of the four alternatives ranged from 5.1 to 7.4. The four alternatives were rated differently: $F(\mathrm{3},\mathrm{172})=\mathrm{10.124}$, p<0.001, η²=0.149. Alternative 2, i.e. an avalanche problem with technical information and advice, was rated highest, and alternative 1, a danger level with an explanation, least informative. A post hoc Tukey test confirmed it: alternative 1 was rated lower than the other three alternatives ($p^{\prime }s=<\mathrm{0.001}$, smallest effect size Cohen's d=0.814). A user's competence had no effect on the ranking of the alternatives: $F(\mathrm{1},\mathrm{172})=\mathrm{1.966}$, p=0.163, η²=0.010. Experience was also non-significant; a user's experience had no effect on the ranking of the alternatives: $F(\mathrm{1},\mathrm{172})=\mathrm{0.469}$, p=0.494, η²=0.002.

Comprehension was good, with all four alternatives yielding overall positive scores, i.e. one-sampled tests for all four alternatives were significantly different from a score of 0 (alternative 1: p=0.015, alternatives 2–4: $p^{\prime }s<\mathrm{0.001}$). Still, the comprehension scores were different for the four alternatives: $F(\mathrm{3},\mathrm{172})=\mathrm{8.188}$, p<0.001, η²=0.120. Alternative 1, a danger level with an explanation, had the lowest comprehension score and was significantly different from the other three alternatives; post hoc Tukey tests had the smallest p=0.021 and the smallest effect size Cohen's d=0.541. Notably, the higher the competence the better the comprehension was: $F(\mathrm{1},\mathrm{172})=\mathrm{7.777}$, p=0.006, η²=0.038. Finally, there was a positive correlation between user ranking and comprehension: ρ=0.2, p=0.008, 95 % CI [.054; .337].

3.4.2 Danger level 4 wet slab scenario

For the danger level 4 wet slab scenario, the average user ranking of the four alternatives ranged from 6.7 to 7.7. All alternatives (danger level, avalanche problem) were ranked high, and there was no statistically significant difference: $F(\mathrm{3},\mathrm{172})=\mathrm{1.787}$, p=0.151, η²=0.030. Also, a user's experience and competence did not influence the ranking (both $F^{\prime }s<\mathrm{1}$).

Comprehension was good, with all four presentations of the scenario yielding overall positive scores. All but alternative 3 had a significant positive score (alternative 3 only marginally: p=0.065, Cohen's d=0.262). The four alternatives did differ: $F(\mathrm{3},\mathrm{173})=\mathrm{4.188}$, p=0.007, η²=0.067. Alternative 3, an avalanche problem with technical information only, received a significantly lower comprehension score than alternative 1 (post hoc Tukey: p=0.037, Cohen's d=0.607), and alternative 4 (post hoc Tukey p=0.006, Cohen's d=0.763). A user's competence had no effect on comprehension: $F(\mathrm{1},\mathrm{172})=\mathrm{0.93}$, p=0.336, η²=0.005. There was also no relationship between user ranking and comprehension: $\mathit{\rho }=-\mathrm{0.095}$, p=0.207.

3.4.3 Comparison

The results show that for the danger level 2 scenario, the three alternatives with the avalanche problems communicate more effectively than the one with the danger level. The user ranking and the calculated comprehension score provide consistent results. For the danger level 4 scenario, on the other hand, the alternatives with the danger level and avalanche problem with travel advice score higher than the two other alternatives. The difference is clearer for the calculated comprehension score than for the user ranking. The alternative with the avalanche problem and technical details seems to communicate least effectively however, it is also possible that users become too careful/conservative and rate factors as important that experts do not, and hence receive a lower score.

4 Discussion

The purpose of this paper was to evaluate how well the avalanche warnings, as provided by NAWS, are communicated to the public, and whether some modes of presenting the warnings are more effective than others in communicating the intended message. We discuss our main findings below: firstly, we discuss the user results that are reported in Sect. 3.2 (avalanche warning); secondly we discuss the results reported in Sect. 3.3 (modes of communication: text versus symbols and pictures) and finally the results from the test of comprehension reported in Sect. 3.4.

4.1 The avalanche warning

Our survey responses suggest that users find it most difficult to assess and manage the snow cover, but that relatively many individuals also find it challenging to manage group dynamics and terrain traps. The fact that so many (both experts, experienced users and novices) struggle with an evaluation of the snow cover and its impact on the avalanche hazard is not surprising. The snow cover is difficult to assess and manage, it can vary considerably over both short time periods and distances (Schweizer et al., 2008). It is created by a complex and dynamic interaction between the atmosphere, the old snow cover, and the ground, and the development of the snow cover over time may create complex structures and properties. Visual surface clues are few and information on the internal structure and properties hard to come by. The difficulty in assessing and managing the snow cover is reflected in what users perceive as most important in the avalanche warning: the avalanche assessment, the avalanche problems and the main message.

A well-functioning avalanche warning needs to translate the complex dynamics and characteristics of the snow cover and avalanche hazard into a clear message that novices and experts find useful and can translate into behavioural implications. However, although the responses to our survey questions suggest that most users demand this type of information, the responses also show that it is challenging to create a warning that will fit all needs. While some users would like to see more detailed information on the type of avalanche problems and geographical distribution of these problems, the characteristics of the snow cover (including spatial and temporal variability), weather patterns, and estimates of uncertainty in the forecast, other users state that the amount of information and detail currently available in the warning is already too high and complex, and that it makes them confused.

Today, the Norwegian avalanche warning describes the avalanche hazard using both symbolic representations, a summary of the behavioural implications, and more lengthy descriptions of the avalanche problem, the snow cover, and the mountain weather. In addition, users have access to snow observations via RegObs. The symbolic representation of the avalanche danger and main message is currently presented at the top of the page, while more detailed information is available lower down or to the side.

Our interpretation of the responses given in the survey is that the avalanche warning should maintain its current structure, with easy-to-grasp information for all users (novices as well as experts) at the top level of the warning, and more detailed and complete information for advanced users, e.g. information about the type of avalanche problem, character, timing, geographical distribution, and reliability of observations, at a lower level. On the other hand, most NAWS experts (77 %) and users (67 %) rated the avalanche problem as the most important element of the warning. This, in combination with several users saying that there is too much and too complex a text (i.e. redundancy), suggests that the avalanche problems should be communicated high up in the warning. A more compact presentation with less information would strengthen the communication efficiency, in particular if the overlap with the avalanche danger assessment text is reduced. We also see several other areas for improvement. (1) The danger level was considered important by many, but by less than 50 % of the users. This may suggest that this element may be better off at a less pronounced place on the page. (2) Many individuals find it difficult to interpret the core sector and elevation diagrams, mainly in terms of identifying safe and unsafe sectors. To remedy this problem, it may prove beneficial to show the danger level and/or avalanche problem in two to three different elevation bands, as is done by several European and Canadian AWS^′, or to use bold red colours (at the risk of confusing it with the red colour used for danger level 4) and clearer fonts. (3) Information on weather is at times repeated in several places in the avalanche warning, e.g. in the weather forecast, the avalanche assessment, the avalanche problems and the snow cover discussion. To improve clarity and readability, it may prove beneficial to remove redundant information about the weather. (4) To increase the usefulness of snow observations, the interface of RegObs may need revision.

4.2 Modes of communication: text versus symbols and pictures

Our empirical analysis shows that most respondents prefer symbols and text to pictures. The preference for text messages may partly be explained by the fact that users have become accustomed to this mode of communication: NAWS has presented the avalanche warnings using text during its 5 years of operation. Another potential explanation is that the names of the avalanche problems are easily communicated verbally – in interviews in the media, during avalanche courses and when discussing the avalanche danger before and during trips. Even though symbols are efficient, text labels are very useful. The EAWS symbols were new to Norwegian users, and the positive rating of these supports the decision to introduce these symbols as a standard in Europe. Unfortunately, it was beyond the scope of this study to test whether the wording of text messages and the symbolic representation of avalanche problems are optimal for communicating the intended message. To do this, more advanced testing is needed.

The relatively low rating of pictures may be explained by the choice of certain pictures, or by the amount of detail in them. Pictures are bound to be taken at a certain location, under certain circumstances, and almost always contain some amount of irrelevant information. As a consequence, the main message may become blurred, and some users may feel that the snow or landscape is not representative of the avalanche forecast or region. More tests of pictures, or even video, could be carried out to explore whether these otherwise very effective media could be used to communicate the avalanche danger, perhaps as a complement to the other parts of the forecast. NAWS has posted pictures and videos of the current situation a number of times on Facebook, and especially the videos have had a large impact measured by the number of views, likes, comments and shares. This suggests that a more elaborate study on the effect of pictures and videos should be carried out. However, the costs of obtaining relevant quality video and pictures may outweigh the benefits.

4.3 Modes of communication: comprehension

We evaluated comprehension of the communicated avalanche warnings by the use of two methods: we first asked respondents to rate how well the avalanche hazard and the associated behavioural implications were communicated, and thereafter tested whether different modes of communication resulted in different “comprehension scores”. All participants evaluated a danger level 2 scenario (wind slab) and a danger 4 scenario (wet slab). In each of these scenarios, the participants were randomly exposed to one out of four alternative descriptions of the avalanche hazard and asked to choose the behavioural implications associated with the warning. The comprehension score was based on a comparison between the choices made by users and a template constructed from the answers made by a panel of NAWS experts.

Our empirical analysis of the subjective rating of the avalanche warnings shows that most users perceive Varsom.no to communicate the avalanche hazard in a good way: 51 % of the users rated the communication of the danger in the level 4 scenarios as 8 or higher, on a scale from 1 to 10 (41 % in the level 2 scenario). These results are consistent with previous studies on user satisfaction (Kosberg et al., 2013; Barfod et al., 2014, 2015) and the conclusions in a recent evaluation of NAWS (Hisdal et al., 2017).

The rating of the different alternatives dependents on the scenario. For the level 2 (wind slab) scenario, alternative 2 (avalanche problem and technical information and advice) received the highest rating, slightly higher than alternative 4 (avalanche problem and advice). Alternative 1 (danger level) was rated the lowest. For the level 4 (wet slab) scenario, it was alternative 1 that received the highest rating but this was not significantly different from any of the other alternatives. These results suggest that the users perceive that they need more detailed information than just a danger level when given a danger level 2, but are highly satisfied with knowing the danger level if it is 4. Notably, for danger level 2 a user's competence mattered when it came to the rating of the alternatives, but not for danger level 4. This suggests that the value of more detailed information about the avalanche problem increase as the user's competence level increase. Level 4 might be a cut-off for most, in terms of making the decision not to enter avalanche terrain. The avalanche-related demographics data from the user respondents (Section A and E) showed that the more competent the users, the more tours they undertake in avalanche terrain. However, quite a few of those without competence or courses are also active ski touring. Most respondents assessed themselves as being competent. Hallandvik et al. (2017) showed that novices assessed the terrain for a specific site as less complex than experts, they weighted information in the avalanche forecast differently, and used different strategies to gather information about the snowpack on a trip. Thus experience and competence matter to a certain degree when communicating avalanche danger. In our study, the sample of respondents may be biased towards experienced and interested users, which may somehow affect the results (e.g. Haegeli et al., 2012).

Our results from the test of the level of comprehension for the level 2 scenario (wind slab) are largely consistent with participants' subjective evaluations: participants score significantly lower on the comprehension score if the only information available is a danger level with a standard explanation (alternative 1). We therefore argue that a simple danger level is not enough to convey the intended warning message on lower danger levels. At this level, the users consider how to travel in avalanche terrain, rather than whether or not to enter avalanche terrain. Rather, in this situation the warning should present the avalanche problem with a reasonable level of detail. Our results do not provide a clear answer to the question of which details are most important in order to communicate the message; indeed, all three alternatives yielded a higher comprehension score than alternative 1.

The results from the level 4 scenario (wet slab) are markedly different from the level 2 scenario, in terms of both comprehension scores and the match between objective comprehension and subjective evaluations. In contrast to the level 2 scenario, users rated all alternatives equally. However, in this scenario, the comprehension score was significantly lower for alternative 3 than it was for alternatives 1 and 4. In other words, leaving out the advice and explanation resulted in a lower comprehension. Note though that all four alternatives yielded positive scores: most users did select the same factors as the experts. One possible explanation for the observed lower comprehension score for alternative 3 is that the technical information caused confusion rather than helped the respondents.

Based on the results of our empirical analysis, our recommendation is that NAWS should carefully assess the importance and priority of details presented with the avalanche problem in order to shorten and simplify the communication of the danger to the different user groups. It is important to consider the redundancy of information. If two or three avalanche problems are presented in the warning, the redundancy between the details in the problems as well as the main message and danger assessment may be considerable. NAWS should consider how to normalize the avalanche warning, in order to avoid repeating the same information several times. Our results also points to the importance of communicating the avalanche problems at a high level in the warning, especially for warnings on lower danger levels. However, both the avalanche situation and the snow cover may be very complex across the warning region and forecasting period. The NAWS warning regions are about 20 times the European average (Engeset, 2013; Techel et al., 2018), and some have rather complex topography and weather patterns. Simplicity may not always be achievable without leaving out important information for the users.

There are several factors that may have affected the results presented in this study, and we would like to linger around them. First, it is important to note that this study targeted recreational users, and not preparedness authority personnel. The needs of novice, advanced, and professional users are likely to differ. While simple symbols and messages are likely beneficial for individuals with less avalanche knowledge, technical details can be of great use for advanced users. Properties such as sector, elevation, size and probability of release are useful when considering which roads or residential areas are exposed to the avalanche danger. We therefore recommend that a more detailed study should be carried out, in order to investigate what is effective to communicate to preparedness users. Second, the Norwegian users were only recently introduced to a national avalanche warning system. This implies that many users have been in the process of acquiring knowledge about how to assess avalanche danger during the period from 2013 to 2018. Other than the avalanche warnings provided by NAWS, few sources for assessing the avalanche danger are available in Norway. Most avalanche courses frequently use the avalanche warnings and avalanche educational resources on Varsom.no. Finally, we only tested a limited set of alternative communication modes to the participants. To fully evaluate how to optimally design the avalanche warning, tests with more variations and scenarios are needed.

5 Conclusions and recommendations

In this study, we investigated how well users perceive the avalanche warnings on Varsom.no (http://www.varsom.no, last access: 9 September 2018) to be communicated, and whether different modes of communication affect the level of comprehension of the hazard at hand. We also identified elements in the avalanche warnings that users perceive to be of greater or lesser importance, easily misunderstood, or missing.

Based on our empirical analysis of the data, we make the following conclusions and recommendations.

1. Redesign core zone and elevation graphics/text. Problem: participants found it difficult to understand whether the avalanche problems were present or absent in coloured sectors. Possible solution: use colours, bolder text and better symbols, and show the danger level and/or avalanche problem in two to three different elevation bands, as is done by several European AWS^′ and in Canada.

2. Less is more. Problem: the amount of text and detail in the warning reduced the motivation to read the warning and made it more difficult for the user to pick up the main message. Possible solution: minimize repetitive information and reduce complexity.

3. Local information matters. Problem: the avalanche warnings are produced for relatively large geographical areas with big spatial variations in the snow cover. Possible solution: use maps to show the parts of the region (subregions or elevation intervals) that are most affected by the avalanche problem(s), or where the avalanche danger is expected to be one value higher or lower than the rest of the region. Maps could show which parts are most affected, by showing the properties creating the avalanche problems, e.g. heavy precipitation, wind, or temperature. NAWS could use sub-regions as a way to provide better information in the textual analysis. NAWS will probably not have information with the required detail to present higher-resolution maps of danger level or avalanche problems yet. Another way could be to present local weather history, and/or snow observations from automatic stations, or to present the snow history by visualizing some manual snow observations as time series.

4. We need to teach snow dynamics. Problem: a very large share of respondents state that they find it most difficult to assess and manage the snow cover. Possible solution: present the avalanche problem, snow cover analysis and avalanche danger assessment in a more systematic and pedagogical manner in order to improve the competence of the users. It should be noted that even the NAWS experts considered the snow cover to be the most difficult factor, suggesting that it is complex to manage for users at all levels.

5. We need to teach people dynamics and terrain traps. Problem: a large share find it difficult to manage others in the group. Many also find terrain traps problematic. Possible solution: use the “avalanche school” to educate users about terrain traps and talk about group dynamics to help users make better choices about whom they choose to recreate with in avalanche terrain.

6. Avalanche problem is important. Problem: the avalanche danger level is not enough for making decisions in avalanche terrain, more detailed information is needed. Possible solution: promote the avalanche problem, especially at danger levels 2 and 3, which also are the conditions most fatalities occur. Streamline the presentation of the avalanche problem according to danger level and reduce overlap with the avalanche danger assessment in order to reduce the complexity for users. Reduce the amount of information to users at higher danger levels. The danger level was rated as important, but somewhat difficult to understand. It is a simple numeric value, but is determined from relatively complex and subjective factors, and is probably difficult for users to understand and use.

7. Keep the EAWS symbols. The users considered the new EAWS standard icons for the avalanche problems to communicate the danger well, although the users were not familiar with the icons.

In conclusion, our study has confirmed that the communication of the avalanche danger on Varsom.no is perceived to be effective by the users. The results of the testing of the effectiveness of different alternatives for communication of level 2 and level 4 avalanche danger suggested that the avalanche problems communicated more effectively than the danger level at lower danger levels. At the higher danger levels, no significant difference was found between the alternatives. The results suggest that a simple danger level is not enough to convey the intended warning message on lower danger levels; rather, the warning should present the avalanche problem with a reasonable level of detail. At higher danger levels, the results suggest that leaving out the advice and explanation resulted in lower comprehension. For danger level 2 a user's competence mattered when it came to the rating of the alternatives, but not for danger level 4. Many users (67 %) and most NAWS experts (77 %) rated the avalanche problem as the most important element of the warning.

Based on the findings in this study, NAWS redesigned the avalanche warning on Varsom.no: the communication of the core sectors was improved (displayed in red signal colour rather than vague grey), a location search function was added, the display of avalanche problems was moved up to just below the main message, the redundancy in information between the avalanche problem, snow cover analysis and avalanche danger assessment was reduced, the region map was relocated down to the bottom of the page and the mountain weather and snow cover analysis was restructured.

Norwegian users, experts and avalanche warnings were used in this study, but we believe the methods and results are important to the wider scientific community and AWS^′ in other countries. The building blocks and communication techniques of the avalanche warnings on Varsom.no follow the standards of EAWS, as Varsom.no and NAWS were developed in collaboration with a number of AWS^′ in Europe and North America.

Our study sheds light on how effectively key information is communicated in avalanche warnings. However, we recommend more studies on communication and impact of avalanche warnings, including in-the-field testing, testing of the use of avalanche problems with regards to people and terrain choices, and further development of methods for quantifying the effectiveness of such communication.