We greatly appreciate you reviewing our manuscript in great detail and providing many helpful comments. Please find below our response (bold) to your comments (italics). 

Overview

This manuscript presents the results of a study that examines the text-based descriptions of avalanche danger in public avalanche forecasts. The study analyzed the text descriptions published in more than 1,000 avalanche forecasts by the national avalanche warning service in Switzerland over eight forecast seasons.

I applaud the authors for recognizing the value of text-based analysis in research of avalanche forecast quality, as this approach has the potential to expand ways to think about and examine the topic. While the study offers valuable insights, it is my opinion that several weaknesses need to be addressed before the manuscript should be considered for publication. Due to my comments about the research objective  and the approach of the analysis, I believe that the manuscript requires major revisions. However, I hope that my comments below offer meaningful starting points for improving the manuscript. In my opinion, the suggested changes will make the paper a much stronger and, therefore, a more impactful contribution.

Major Comments

Link between theoretical background and analytic approach

I appreciate the introduction of the semiotic triangle as a conceptual framework for the task of avalanche forecast production. As the authors point out (lines 41-44), the semiotic triangle is helpful in that it tracks the process from an avalanche situation to forecaster interpretation to a communication. However, I am not able to see the connection from this conceptual framework to the methodological approach. To make this connection stronger, I recommend that the authors revise their introduction to present their research questions and objective in a more accurate way.

This is a fair point, made by both reviewers. We think the ideas of the semiotic triangle are useful in transferring ideas from other fields to avalanche forecasting that are relevant to the ways in which we communicate and deal with information. We will therefore provide a more-in-depth introduction of the semiotic triangle, and explain how it applies to avalanche forecasting (by use of examples and figures), how it relates to previous research, and where our study is situated within the context of the triangle. We will strengthen the link between this concept and data, methods and results.

Clarifying the stated objective

To add more detail to the comment above, the stated objective of this study requires further clarification. If the objective of the research is to demonstrate the value of text-based analysis to avalanche forecast research (lines 54 and 66-68), the authors need better situate and justify their rationale for the study design and analysis within the body of literature on text-based methodologies. As the methods section does not include any citations to support the methodological approach beyond validating the inter-rater agreement rate (lines 169-170), the authors need to provide more adequate support to ensure that the study is well-grounded and that the reader can see how it makes a contribution to the stated objective.

If this extends beyond the possibilities of the current analysis, I recommend that the authors reword the objective to make it clear that the goal is to contribute to an official translation of terms characterizing key factors of avalanche hazard rather than to demonstrate the value of text-based analysis in avalanche research.

The objective of the study is to demonstrate how avalanche danger is described (and whether this is in line with definitions) by means of text-based analysis. Thus, showing the value of text-based analysis to avalanche forecast research as a way to explore the text, which is the least-structured part of avalanche forecast products, is an objective. We will therefore provide more background on methodologies used for text-based analysis, and where our study design is situated.

RQ1: An analysis of forecaster agreement may not represent language use

While important insights emerge from the analysis of RQ1, the task does not replicate the forecasting workflow and the implications to the avalanche forecasting process require re-examination. There is a crucial difference between the analytic exercise designed to examine RQ1 (lines 26-29) and the forecasting process outlined in the semiotic triangle. In contrast to the semiotic triangle, the analytic exercise does not replicate the forecasting task of moving from an avalanche situation to an interpretation and subsequent communication symbol. Rather, it orders this process in reverse, whereby the forecaster is tasked with matching a communication symbol to a corresponding key factor in an avalanche situation. Thus, the analytic approach does not examine how language is used by forecasters in the context of how forecasts are produced, which is what RQ1 might suggest given its current wording (i.e., “how do forecasters use language….”) (lines 75-76). A more precise wording of RQ1 might read, “how well do forecasters agree on the meaning of key phrases....” 

We agree, regarding the formulation of RQ1, we will reword it according to the suggestion. In this context, it is of note, however, that the semiotic triangle is bi-directional: 

On the forecaster side: from an avalanche situation (referent) to the text (symbol). This, however, may go back-and-forth, if for instance several forecasters work on the same forecast.

On the user side: from the text to the avalanche situation, which the user expects reading the forecast. Users will compare the text (symbols) to the observed conditions, which they will translate back to symbols themselves. We don’t investigate how users interpret forecasts in this paper, but the framing is important in understanding our approach and for motivating future work.

We will explain these concepts in greater detail, integrating the semiotic triangle better in the manuscript overall.

RQ2: Establishing a hypothesis

Research question 2 (lines 133-135) involves analyzing how the classified text descriptions correlate across differences in avalanche danger. The authors distinguish avalanche danger according to the different levels of severity as classified by the European Avalanche Danger Scale and according to dry-snow versus wet-snow conditions. The analysis examines a measure of the completeness of trigger, likelihood, and size information across differences in avalanche danger as well as examines their content distinguished by natural and additional load triggers; few, several, or any triggering locations; and sizes 1, 2, 3, 4, and 5.

To establish a starting point for expected outcomes, the authors need to provide a reference to the full European Avalanche Danger Scale in the main body of the manuscript. Based on the formal definitions of the various levels, what differences in completement and content, if any, would be reasonable to expect? Providing this background and hypotheses would better situate the results in terms of how they confirm or contrast existing expectations. This would help to expand the discussion of the value of the danger description and the potential reasons for the observed variabilities.

We will refer to the European Avalanche Danger Scale (EADS, EAWS, 2018) in the main body of the paper, and provide the EADS in the Appendix. We will explain in more detail how forecasters in Switzerland use this scale in terms of snowpack stability (or sensitivity to triggers), and the likelihood (Data Section). We will discuss how the results match the description in the EADS in the Discussion section. 

Similarly, the authors need to include their rationale for including and differentiating dry-snow versus wet-snow avalanche types. Why do they make this distinction? Are any avalanche conditions excluded from this distinction? And finally, how is information completeness and content expected to differ across these conditions? Providing this background and hypotheses would better situate the results in terms of how they confirm or contrast existing expectations. 

In the Swiss forecast, dry-snow conditions are essentially always summarized by a danger rating, with the danger description describing this danger rating. Wet-snow conditions, on the other hand, are often mentioned as a secondary problem. In Switzerland, this means that the danger resulting from this secondary problem is at most as high as the danger level communicated with the primary problem, but may often be lower. Furthermore, and this is not covered in the EADS, wet-snow or gliding avalanches are rarely human-triggered, which contrasts to dry-snow avalanches. 

Communicating Uncertainty

As the authors detail in their explanation of the semiotic triangle (line 40), a key aspect of the cognitive task in avalanche forecasting is that forecasters may work through the semiotic triangle with incomplete information, which produces various levels and sources of uncertainty. Is it possible that situations of extreme danger might have different levels or sources of uncertainty than situations of moderate danger and might explain some of the resulting patterns in the analysis? 

The authors do highlight this possibility in the discussion section (lines 338-339). However, as the element of incomplete information was pre-defined in the semiotic triangle, this very limited mention of it in the discussion seems underdeveloped and incomplete. The manuscript would benefit from elaborating on the role of incomplete information as it currently leaves a lot of questions open.

These are useful points that we will incorporate in the revised discussion. There are various sources of uncertainty in avalanche forecasting as for instance the uncertainty related to the availability of relevant data (or the lack thereof ) and uncertainty related to the avalanche conditions (thus related to the danger level). We will elaborate on this when revising the discussion by addressing these sources of uncertainty and how they may be expressed in the language in the danger description but also in the EADS. 

RQ2: The inclusion and exclusion of phrases

Through the analysis of RQ1, text phrases that did not produce high levels of agreement among forecasters regarding key factors were subsequently excluded from additional analyses. This begs the question: what themes were encompassed by these ambiguous phrases? Could these phrases also offer valuable insight into avalanche forecast quality? I recommend that the authors consider conducting further analysis of the excluded phrases. The results could then be incorporated into the analysis of RQ2 for a more robust analysis. Is it possible that the themes encompassed by the ambiguous phrases might correlate with specific hazard conditions and might provide insights into what forecasters deem important to danger descriptions beyond key phrases?

This is an important point, and we will try to make clearer the iterative annotation process we used, typical of such text analysis. For example, we will illustrate phrases which were thought to represent one of the key factors describing avalanche hazard in initial annotation. For instance, text describing avalanches releasing deep within the snowpack or weak layers existing close to the snow surface, could be interpreted as being related to avalanche size. Deep within the snowpack and close to the surface were therefore assigned a relation with avalanche size in the first annotation round. However, in the second round, when annotators were specifically asked to assign a size class (or two), none could do so. Since we discarded terms where agreement was poor between annotators, we cannot go beyond giving examples, since by definition these phrases were then not labelled.

Additionally, does it make sense to include phrases in the analysis that were never used in a bulletin? This should be addressed in the limitations section. 

The annotation was performed at the level of the entire set of phrases, not the list of phrases used in the forecasts. For completeness, these phrases are shown in the Appendix Tables. A phrase not being used may either be due to it being typical for a rare situation (for instance describing danger level 5-Very High) or because forecasters are not in full agreement using this phrase as suggested in the EADS (for instance a single mountain climber representing a high additional load). 

Implications for users of avalanche forecasts

Line 338: “Leaving out information, for example the likely triggers or size classes of avalanches expected for danger level 2-Moderate, may, for forecasters, actually convey information about the situation.” Please elaborate on this. Maybe provide an example.

We will provide an example in the manuscript. This may be, for instance: the findings show that the expected occurrence of natural avalanches is consistently mentioned. At lower danger levels, when no "natural" avalanches are mentioned, a forecaster would probably understand that an additional load is required to release avalanches. (A user, of course, may not be aware of this. But that is a different issue.)

There are various papers, such as Lazar et al. (2016), Statham et al. (2018), and Clark (2018 and 2019) that shed light on consistencies or inconsistencies among avalanche forecasters. I think it would be useful for this paper to include these ISSW papers in the discussion.

Thank you for pointing these out. We will incorporate these when we discuss consistency.

The discussion does not include any recommendation for avalanche forecasters or the Swiss avalanche bulletin system (e.g., use of phrase catalogue). While there is a brief mentioning of the graphic display of avalanche hazard information in Canadian avalanche bulletins, a critical discussion of how the graphical approach and/or the conceptual model of avalanche hazard (Statham et al., 2018) can address the identified challenges is missing. I believe that a broader discussion would make this a more useful paper for the global avalanche safety community.

We will add a subsection in the Discussion, where we will be more specific about possible recommendations (e.g. a Section 5.3. Implications to forecasters, or similar).

Limitations

Given that the use of the sentence catalogue seems to be very specific to the production of the Swiss avalanche bulletin, I don’t think it is realistic to expect that the results would be transferable to other warning services. I believe that the focus on Switzerland should be clearly stated in the research objectives. This means that this aspect likely does not need to be mentioned in the limitations section.

The results are clearly specific to the Swiss forecast even though the sentence catalogue is used by five warning services in Europe. We will emphasize that we focus on Switzerland in the research objective, but we still believe that not being able to transfer the findings easily to other forecast products is a limitation. 

Per my earlier comment on the the inclusion and exclusion of phrases in the analysis, I believe that this should be addressed in the limitation section.

We will add a comment in this regard.

Minor Comments

Triggering terminology

I find the terms used to describe the key factors related to triggering avalanches to be wordy and confusing (i.e. triggering leve, triggering spots frequency, and triggering spots location). I think the following terms from the Conceptual Model of Avalanche Hazard (Statham et al., 2018) offer a clearer delineation of these key elements: trigger type, sensitivity, and spatial distribution. These elements are then combined to form the likelihood of avalanches, whereas the size classifications offer an ordinal representation of consequence. I recommend the use of these terms as it strengthens the connection to well-established definitions of key factors within risk science. 

As the basis for forecasting in Europe and Switzerland is primarily the EADS, we will stick with well-established terms currently in use in Europe (current definitions and descriptions provided on the EAWS webseitewww.avalanches.org like EADS, avalanche problems, and so on). We are aware that there are different terms in use in the CMAH, and we mention this in several places in the manuscript. The contributing factors to avalanche hazard in both the EADS and the CMAH are the same, although they may be called something different. Particularly what is referred to as “spatial distribution” in the CMAH, does not exist in the same way in the EADS. The EADS describes primarily the number or frequency of hazardous spots / triggering locations / avalanches, the CMAH mixes terms which are more related to frequency (isolated, widespread) and location (specific). Because we acknowledge that both the number of potential triggering locations as well as their actual location is relevant information, we split the “spatial distribution” into frequency and location.

We will better explain why we split into frequency and location information. Incidentally, this usage of terms is a good example of the semiotic triangle, since the symbols (words in this case) used by different avalanche services can be different, but may represent similar or overlapping concepts. 

Introduction

The introduction is fully focused on European avalanche bulletins. Since the authors refer to non-European avalanche bulletin formats in the discussion section, I think the manuscript would benefit from including a more in-depth description of how the information presentation in the Swiss bulletin compares to others. For example, the text information (avalanche activity, snowpack conditions, weather) included in Canadian and US bulletin offers more detailed insight about conditions than the text included in Swiss bulletins. Furthermore, a broader description of the context in the introduction will make the paper more relevant for a wider audience.

This is a useful point and we think it could strengthen the paper’s introduction. Using examples, we will provide an overview of how the three contributing factors of avalanche hazard are addressed in forecast products issued by different warning services (i.e. graphics, bullet list, danger description), how they are referred to, and whether they are compulsory elements or not. This will highlight more clearly differences between the Swiss forecast and other forecasts.

Line 63: Clark (2019) examined the link between the likelihood and expected size of avalanches with the avalanche danger rating. The manuscript does not accurately describe this research.

Thank you for pointing this out. In our reading of Clark’s work he explored the severity of the avalanche problem, described by likelihood of avalanches and expected size of avalanches (for each avalanche problem type separately),in relation to the avalanche danger rating. This latter part was indeed missing. We will revise the text accordingly.

Line 106: At the core of the danger description “in Switzerland”…

Will be done.

Discussion

Line 346: I do not understand how the results of the analysis suggest that “communication of non-extreme situations is critical”. This statement requires elaboration.

We will explain this in our revision.

Technical Comments

Thank you for pointing out these issues. We will address these when revising the manuscript.

References

Clark, T., & Haegeli, P. (2018). Establishing the link between the Conceptual Model of Avalanche Hazard and the North American Public Avalanche Danger Scale: Initial Explorations from Canada. Proceedings of the 2018 International Snow Science Workshop, Innsbruck, Austria, 1116-1120. https://arc.lib.montana.edu/snow-science/item.php?id=2718

Clark, T. (2019). Exploring the link between the Conceptual Model of Avalanche Hazard and the North American Public Avalanche Danger Scale. M.R.M. research project, no.604, Burnaby, BC. School of Resource and Environmental Management, Simon Fraser University. https://summit.sfu.ca/item/18786

EAWS: European Avalanche Danger Scale (2018/19), https://www.avalanches.org/wp-content/uploads/2019/05/European_Avalanche_Danger_Scale-EAWS.pdf, last access: 14 Feb 2020, 2018.

Lazar, B., Trautman, S., Cooperstein, M., Greene, E., & Birkland, K. (2016). North American Danger Scale: Are Public Backcountry Forecasters Applying It Consistently? Proceedings of the 2016 International Snow Science Workshop. Breckenridge, CO, 457-465. https://arc.lib.montana.edu/snow-science/item.php?id=2307

Statham, G., Haegeli, P., Greene, E., Birkeland, K., Israelson, C., Tremper, B., & Kelly, J. (2018). A conceptual model of avalanche hazard. Natural Hazards, 90(2), 663-691. https://doi.org/10.1007/s11069-017-3070-5

Statham, G., Holeczi, S., & Shandro, B. (2018). Consistency and accuracy of public avalanche forecasts in Western Canada. Proceedings of the 2018 International Snow Science Workshop, Innsbruck, Austria, 1491-1495. https://arc.lib.montana.edu/snow-science/item.php?id=2806

We greatly appreciate you reviewing our manuscript and providing helpful, constructive comments. Please find below our response (bold) to your comments (italics). 

The manuscript describes an effort to examine the consistency of text descriptions of the avalanche danger with the avalanche danger rating. The authors use a unique, or at least uncommon, dataset to address two research questions about the use of text to convey key elements of the public avalanche forecast with the danger level and if the use is consistent at different danger levels. The author’s approach is interesting and it is a unique application of these ideas in this context. Their results are also interesting, although maybe not surprising, and worthy of documentation in the literature. The structure of the manuscript and the writing within it has room for improvement and I do not feel it should be published in its current form. The revisions required are more than minor and thus my recommendation is for the authors to complete major revisions before other reviewers evaluate the work.      

The manuscript reads little like it was extracted from a larger document. There is both too much detail and use of very specific terms (like phrase_option) in some sections and not enough background and explanation in others (see note on semiotic triangle below). This issue runs through many parts of the manuscript. It makes interpreting some of the figures difficult as they are dense with information, but don't provide the reader with much guidance on what is included to provide a wholistic view of the work and what is critical to understanding it (Table 3 is a good example). The Methods section includes descriptions of which author produced different parts of the analysis, although I’m not sure if this is important for me to know and if so why. 

Thanks for this useful pointer. We will revise the manuscript taking care to make the level of detail more appropriate and more “reader friendly”.

The Data section explains a lot about how the avalanche forecasts are issued and identifies elements of the products, but it does not paint a clear picture of how the structure of the phrase catalog informs or affects the analysis or results. 

We will be more clear about this. The catalogue of phrases impacts the forecast product as all forecasters use the same set of words. It also impacts the analysis as the number of words, though quite large, is finite. Only this combination makes the research possible.

The Results section is very dense with a lot of information, but sorting through it is challenging (thank you for the summaries by danger level!).

We intend to provide a summary table showing the key results (as currently on lines 240-244 and 257 - 261, together with the description used in the European Avalanche Danger Scale, EADS; EAWS, 2018) to allow a better comparison between our findings and the definitions in the danger scale.

I am both intrigued and confused by the appendix, which has lots of information that is interesting but I’m not sure how important it is for understanding or documenting the work.

We consider the appendix as complementary information: for researchers who intend to reproduce such work, but also for those who understand both German and English. Importantly, the analysis was fully conducted on German text, and we consider it important to make clear the (possible) implications of translation, without impacting on the readability of the main body of the paper. We will provide more explanation with the tables in the Appendix to facilitate their understanding.

The text in the manuscript could be improved. It is free of errors and typos. However, the writing does not always help the reader focus on the most important issues faced by the researchers or highlighted by the research. An example is paragraph 40. It is rich with ideas and constructed so it is hard for the reader to mover trough it smoothly.  

We will rephrase this paragraph. We intend to provide a figure highlighting the concept of the semiotic triangle with regard to avalanche forecasting, giving an example, and highlight more clearly where previous research and our study fits in.  

Here are a few specific comments:

Title – The subtitle feel closer a description of what is contained in the manuscript. The main title is quite broad and I do not feel like it really helps the reader know what to expect within the article. I suggest they authors refine the title and subtitle structure.

We will reword the title taking this suggestion into account.

Terminology – The authors use some terms in ways that are consistent with previous work, but also that are probably not part of current general use for most of the readers they are trying to reach. 

We will introduce the terms more clearly, including their origin. We will stick primarily to European terms (i.e. European Avalanche Danger Scale) as these are still the binding guidelines in European avalanche forecasting. However, we will link these terms to the Conceptual Model of Avalanche Hazard (Statham et al., 2018), which is increasingly used in Europe as well. 

They do dedicate a paragraph in the introduction to explain the linguistic model they are applying, which is admirable. However, I found the layout and use of some terms in the manuscript confusing. One example is symbol. This is important to the model the author’s use and it is also commonly used in warning communication. The authors use it in the context of the linguistics model and also to refer to graphical elements. The paragraph on the linguistics model contains a lot of good information, but the liberal use of parenthetical phrases makes the material difficult to digest. My suggestion is to do one of two things: remove the discussion of the semiotic triangle and associated ideas and focus on the consistency issues in the stated research questions, or expand the discussion of the semiotic triangle and associated concepts. If this concept is integral to the work, maybe it deserves its own section with a clear explanation. Applying this concept to avalanche forecasts is certainly interesting, but I am not sure if it is fundamental to understanding the work. To me the work described in this manuscript focuses on issues of forecast consistency (consistency of elements within a forecast). If the authors opt to keep the concepts of the semiotic triangle, I suggest they take some time in the proposed section to clearly define how the terms and concepts in this linguistic model are represented in the avalanche forecasts they are analyzing.

After careful consideration of your recommendation, we decided to stay with the semiotic triangle as our theoretical framework. However, besides introducing the concept in greater detail and with regard to avalanche forecasting and avalanche forecast research, we will highlight more clearly that consistency in the use of the terms, for instance when compared to the EADS, is another important objective in our study.

Focus of the results – This work is very specific to the public avalanche forecasts in Switzerland. The authors acknowledge this in the title. In many other parts of the manuscript, the text in not as specific and often is phrased in a way that makes the reader feel like they are learning about avalanche forecasts in a broad sense. This could and should be improved. The authors should focus on Swiss products. This study would probably not be possible with a broader dataset. This provides the authors opportunity to focus on specific aspects of the dataset and interpret the results in a realistic and targeted fashion. 

We will take up this suggestion and be more explicit that we are analyzing Swiss forecasts.

Last sentence of abstract – “Our results provide data-driven insights that could be used to refine the ways in which avalanche danger could and should be communicated, especially to recreationalists, and provide a starting point for future studies on how users interpret avalanche forecasts.” These are very important issues and certainly worth studying and improving. However, I don’t see how that is done in this work. The work focuses on the internal consistency within an avalanche forecast – text descriptor and avalanche danger. It really doesn’t tackle how avalanche danger or the threat to a person could or should be communicated. Just the consistency within the public product in Switzerland. This is a study of how avalanche danger IS being communicated. Given that internal consistency is an important element of any warning product, this work could be a measure of the effectiveness of that product from the producer’s perspective (ie consistent elements are important and reduce the potential of confusing of the target audience). However, there is no measure of how the target user is accepting, comprehending, or effectively applying the warning product.

We will rephrase the abstract and carefully go through lines 345 - 350, where we discuss which findings may potentially impact users of the avalanche forecast.

References:

EAWS: European Avalanche Danger Scale (2018/19), https://www.avalanches.org/wp-content/uploads/2019/05/European_Avalanche_Danger_Scale-EAWS.pdf, last access: 14 Feb 2020, 2018.

Statham, G., Haegeli, P., Greene, E., Birkeland, K., Israelson, C., Tremper, B., & Kelly, J. (2018). A conceptual model of avalanche hazard. Natural Hazards, 90(2), 663-691. https://doi.org/10.1007/s11069-017-3070-5