A neural network model for automated prediction of avalanche danger level

Snow avalanches cause danger to human lives and property worldwide in high-altitude mountainous regions. Mathematical models based on past data records can predict the danger level. In this paper, we are proposing a neural network model for predicting avalanches. The model is trained with a quality-controlled sub-dataset of the Swiss Alps. Training accuracy of 79.75 % and validation accuracy of 76.54 % have been achieved. Comparative analysis of neural network and random forest models concerning metrics like precision, recall, and F1 has also been carried out.

1 Introduction

Accurate prediction of snow avalanches can help ensure people's safety in snow-covered regions. Many countries still depend on human experts to analyse meteorological data to forecast avalanche warnings.

The major hurdle in developing machine learning models is the lack of sufficient and reliable data. This issue has been resolved to a great extent by the WSL Institute of Snow and Avalanche Research, Switzerland, by collecting 20 years of data in avalanche forecasting. This dataset has further been refined with quality control by experts. The dataset combines different feature sets with meteorological variables.

This unique dataset has enabled experimentation with machine learning models like neural networks and has compared its performance with the random forest machine learning technique.

This paper is organized as follows. Related literature is briefly overviewed in Sect. 2. The dataset used for the training of neural networks is described in Sect. 3. After that, in Sect. 4, we explain the neural network model, tuning of hyperparameters, and evaluation metrics. Random forest machine learning method details applied to the same dataset are described in Sect. 5. Results from both methods are compared and analysed in Sect. 6. The paper is concluded in Sect. 7.

2 Related work

Many countries face snow avalanche hazards with snow-clad mountains. It affects people, facilities, and properties. The impact of snow avalanches on living, work, and recreation in Canada is well documented (Stethem et al., 2003). Every country generally follows its own avalanche classification system. However, in this work, we will follow the European Avalanche Danger Scale (EAWS, 2018).

A comprehensive dataset with the meteorological variables (resampled 24 h averages) and the profile variables extracted from the simulated profiles has been created (Pérez-Guillén et al., 2022a). Weather station data of the IMIS (Intercantonal Measurement and Information System) network in Switzerland for dry-snow conditions are further quality controlled for creating 29 296 records. Each record has 30 variables.

The benefits and challenges of using machine learning and AI for avalanche forecasting in Norway and Canada have been discussed in detail (Horton et al., 2020). Also, machine learning algorithms like the random forest has successfully been used for the prediction of snow avalanches in the region of the Swiss Alps (Pérez-Guillén et al., 2022b). The random forest technique has also been used for forecasting snow avalanches in the Himalayan region (Chawla and Singh, 2021). Numerical modelling techniques (Singh et al., 2005) and artificial neural networks (Singh and Ganju, 2008) have been used for prediction of snow avalanches in the Indian Himalayas. Deep learning methodology has also recently been explored for predicting snow avalanches in Iran (Chen et al., 2022) and Kazakhstan (Blagovechshenskiy et al., 2023).

3 Dataset

In this paper, the public dataset provided by Envidat, a Swiss organization, is used. This data are verified and supported by the Swiss Data Science Center (grant C18-05 DEEP snow). More than 20 years of data for avalanche forecasting in the Swiss Alps are provided. Data cover the Swiss winters from 1997–2017. The data are collected from 182 snow stations and are used by the Swiss avalanche warning service.

The dataset includes the meteorological variables (resampled 24 h averages) and the profile variables extracted from the simulated profiles. The dataset contains the danger ratings published in the official Swiss avalanche bulletin using SNOWPACK simulations. The SNOWPACK simulations provide two different output files for each station: (i) time series of meteorological variables and (ii) simulated snow cover profiles.

Table 1 Thirty meteorological variables in four categories (measured, extracted, profiled, modelled) (Pérez-Guillén et al., 2022a) used to develop the neural network model.

This study uses measured, extracted, profiled, and modelled variables. The entire dataset is pre-processed to remove missing and duplicate values. Further, the dataset is normalized by using a scalar function. Also, categorical variables used for danger levels are replaced with numerical representation for compatibility with neural network training software. In addition, some of the irrelevant columns with information about the date, the sector ID, the name of the sector region, the elevation width, the elevation station, and warning are removed from the dataset. The 30 variables used for the training of the neural network model for predicting snow avalanches are shown in Table 1.

4 Proposed neural network: NNM-1

Neural network models allow for the modelling of complex nonlinear relationships between the multiple input and output variables. It is a network of input, output, and intermediate layers (Fig. 1). The outputs are obtained by a linear combination of the weights with inputs. The weights are selected using a “learning algorithm” that minimizes a “cost function” (Bishop, 2005).

Figure 1 Multilayer neural network model.

This study uses multilayer feed-forward networks, where each layer of nodes receives inputs from the previous layers. The outputs of the nodes in one layer are inputs to the next layer. For example, the inputs into the hidden neuron in Fig. 1 are combined linearly to give the following output:

$$\begin{array}{}\text{(1)}& z_j=b_j+\sum w_{ij}{x}_j,\end{array}$$

where z_j denotes the hypothesis of parameters w and b, and x_j denotes the features in the training set.

A nonlinear function modifies the above outputs of nodes before being used as inputs by the next layer. The parameters b_j and w_ij are learned from data. The number of hidden layers and nodes in each hidden layer is specified in advance.

Training of artificial neural networks, also known as supervised learning, involves adjusting weights until the model is properly fitted with labels indicating the avalanche danger according to European norms. A total of 30 input variables are used for training the network. The avalanche threat is categorized into five zones as follows:

•  

  1 – Low

•  

  2 – Moderate

•  

  3 – Considerate

•  

  4 – High

•  

  5 – Very high

The model's performance across the training dataset is described by a loss function which computes the difference between the trained model's predictions and the actual incident instances. The loss function would be very high if the gap between expected and actual results is too large. The loss function gradually learns to lower the prediction error with the optimization function (Bottou, 1991). A multi-class classification cost function is used for avalanche prediction for each danger level. The average difference between the probability distributions that were anticipated and that occurred is calculated.

$$\begin{array}{}\text{(2)}& \text{Cross entropy loss}=-{\sum }_{i=\mathrm{1}}^{\mathrm{5}}{y}_i\cdot \log {\widehat{y}}_i\end{array}$$

Equation (2) computes cross entropy loss using the target and predicted danger levels.

In our scenario, the output layer is set up with five nodes (one for each danger level). The SoftMax activation function is used to compute the probability for each danger class z_j:

$$\begin{array}{}\text{(3)}& \text{SoftMax}\left(z_j\right)={\displaystyle \frac{e^{z_i}}{{\sum }_j{e}^{z_j}}}.\end{array}$$

Equation (3) transforms the raw outputs of the neural network into probabilities (Christopher, 2005).

The gradient descent method has been used to update the weights and bias through backpropagation. The Adam (adaptive moment estimation) optimizer is used for optimization. It performs the search process using an exponentially decreasing moving average of the gradient.

The performance of a neural network mainly depends on the number of hidden layers and the number of neurons in the respective hidden layer. Table 2 shows the range of hyperparameters used for testing different neural networks. The upper limit on the number of neurons has been set according to the number of raw variables in the original dataset.

Table 2 Range of hyperparameters used for testing neural networks.

Figure 2 Effect of dropout on training and validation accuracy. (a) NNM no. 1 without a dropout; (b) NNM no. 1 with a dropout of 0.2, 0.1, and 0.1 on three hidden layers; (c) NNM no. 2 without a dropout; (d) NNM no. 2 with a dropout of 0.2, 0.1, and 0.1 on three hidden layers; (e) NNM no. 3 without a dropout; (f) NNM no. 3 with a dropout of 0.2 and 0.1 on two hidden layers; (g) NNM no. 4 without a dropout; and (h) NNM no. 4 with a dropout of 0.1, 0.1, and 0.1 on three hidden layers.

After exhaustive testing of neural networks with hyperparameters (Table 2), networks are ranked according to training accuracy (Table 3). However, after plotting of training and validation curves (Fig. 2a, c, e, g), it is observed that validation accuracy was reduced though training accuracy was increased, thus indicating over-fitting of the neural network models. To address this problem, dropout layers are included and tested with different dropouts. Figure 2b, d, f, h shows that dropout regularization successfully resolved over-fitting and significantly improved validation accuracy. Maximum validation accuracy is observed for NNM no. 3 with a dropout of 0.2, 0.1, and 0.1 on three hidden layers (Fig. 3).

Table 3 Neural network models ranked according to training accuracy with a learning rate of 0.001 for 100 epochs and a batch size of 64.

Figure 3 Improvement in validation accuracy of neural network models with dropout regularization. Maximum validation accuracy was achieved with NNM no. 2 with a dropout of 0.2, 0.1, and 0.1 on three hidden layers.

Figure 4 Confusion matrix for NNM-1.

The proposed neural network architecture based on the above study is shown in Table 4. It has three hidden layers and has been trained for 100 epochs. The model achieved a training accuracy of 79.75 % and a validation accuracy of 76.54 %. A confusion matrix for the proposed NNM-1 is shown in Fig. 4 predicting a considerate danger level; out of 1000 cases, 806 cases are true positives, and 194 cases are false positives.

Table 4 Proposed neural network model (NNM-1) architecture.

Evaluation matrix for NNM-1

Table 5 shows the various evaluation metrics like accuracy, precision, recall, and F1 score for the neural network model. The proposed neural network correctly predicted 76 classifications for every 100 forecasts made. The macro- and weighted averages of precision, recall, and the F1 score are shown in Table 5. The macro-average is computed without considering the proportion of labels in different classes of danger levels. It may be noted that the weighted average takes into account a low number of labels for high and very high danger level classes. The proposed neural network model achieved a macro- and weighted average F1 score of 0.69 and 0.75, respectively.

Table 5 Parametric evaluation metrics for the proposed neural network model.

5 Random forest

A random forest is a meta-estimator that employs averaging to increase predictive accuracy and reduce over-fitting after fitting numerous decision tree (DT) classifiers to different dataset subsamples (Kroese et al., 2019). A subset of the training data is randomly chosen by the random forest classifier to construct a set of decision trees. It simply consists of a collection of decision trees from a randomly chosen subset of the training set, which is subsequently used to decide the final prediction. The confusion matrix for the random forest classifier is shown in Fig. 5. The dataset (2920 records) used for validating the neural network model is applied to the computing performance matrix. Several decision trees make up the random forest model, which is trained with the Classification and Regression Tree (CART) algorithm. Table 6 shows the various evaluation metrics like precision, recall, and F1 score for the random forest model.

Figure 5 Confusion matrix for evaluating random forest classification.

Table 6 Evaluation for the random forest model (RF-A).

6 Results and discussion

Testing of the proposed model has been carried out with 2920 records for which ground truth labels are available. The data for the high and very high avalanche threats are less compared to low, moderate, and considerate threats. This scenario in a dataset where samples of data in one class are much higher compared to that of the other class is a skewed dataset. In this case, the higher data sample class (low, moderate, and considerate avalanche threat) becomes the major class. The class consisting of relatively fewer data samples (high and very high avalanche threat) is labelled as a minor class. Hence, the overall neural network performance is affected, thereby generating less accurate results for the minor class.

We trained multiple neural network models with a variety of hyperparameters. The model NNM-1 (Table 4) used for the comparative analysis is without over-fitting and has maximum validation accuracy. Another random forest model RF-A (Möhle et al., 2014) is also tested with the same datasets. Both models (NNM-1 and RF-A) achieved the same overall accuracy (0.76) as RF-1, which is slightly less than RF-2's accuracy (0.78). F1 scores for low, medium, and considerate classes are equal for NNM-1 and RF-A models (Table 6). However, the low F1 value for the high and very high class for NNM-1 and RF-A is attributed to skewed data distribution. Weighted average values (Table 5) are more appropriate compared to macro-average values, as these consider a low number of labels for classes 3 and 4.

7 Conclusion

A neural network model to predict avalanche danger levels has been developed. The model is validated by using 20 years of meteorological measurements and extracted and modelled variables of the Swiss Alps. Extensive testing has been carried out for tuning hyperparameters, like the number of hidden layers and neurons. The data used for testing the neural network model are also applied to the random forest model for the evaluation of performance metrics (Table 7). The developed model has achieved a training accuracy of 79.75 % and a validation accuracy of 76.54 %, which is the same as RF-1 and RF-A but with 2.56  % less accuracy than RF-2.

Table 7 Various parametric values of some existing models and proposed NNM-1 for snow avalanche prediction.

Appendix A: Formulas for the evaluation metrics

Performance indicators like accuracy, precision, recall, and F1 score are used for assessing the effectiveness of the avalanche prediction model. The notations used are

• TP (true positive), the number of points that are positive and predicted to be positive;

• FN (false negative), the number of points that are positive but predicted to be negative;

• FP (false positive), the number of points that are negative but predicted to be positive;

• TN (true negative), the number of points that are negative and predicted to be negative.

The accuracy of classification is the ratio of correct predictions to the total number of input samples.

$$\begin{array}{}\text{(A1)}& \text{Accuracy}={\displaystyle \frac{\text{Number of predictions}}{\text{Total number of predictions}}}\end{array}$$

Precision is the total number of successfully classified positive classes to the total number of anticipated positive classes.

$$\begin{array}{}\text{(A2)}& \text{Precision}={\displaystyle \frac{\text{TP}}{\text{TP}+\text{FP}}}\end{array}$$

Recall is the total number of correctly classified positive classes to the total number of positive classes.

$$\begin{array}{}\text{(A3)}& \text{Recall}={\displaystyle \frac{\text{TP}}{\text{TP}+\text{FN}}}\end{array}$$

The F1 score is the harmonic mean of precision and recall. Mathematically, it can be expressed as

$$\begin{array}{}\text{(A4)}& \text{F1}=\mathrm{2}\cdot {\displaystyle \frac{\mathrm{1}}{\frac{\mathrm{1}}{\text{Precision}}+\frac{\mathrm{1}}{\text{Recall}}}}.\end{array}$$

The formulas used for calculating the macro- and weighted average are as follows:

$$\begin{array}{}\text{(A5)}& \text{Weighted average}={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^{\mathrm{5}}{w}_i{x}_i}{{\sum }_{i=\mathrm{1}}^{\mathrm{5}}{w}_i}},\end{array}$$

where w_i denotes the weights of the five classes, and x_i denotes the value, and

$$\begin{array}{}\text{(A6)}& \text{Macro-average}={\displaystyle \frac{{\sum }_{i=\mathrm{1}}^{\mathrm{5}}{x}_i}{\mathrm{5}}},\end{array}$$

where x_i denotes the value, and 5 is the number of the target variables.