To date, many rockfall hazard assessment methods still consider qualitative observations within their analysis. Based on this statement, knowledge and expertise are supposed to be major parameters of rockfall assessment. To test this hypothesis, an experiment was carried out in order to evaluate the influence of knowledge and expertise on rockfall hazard assessment. Three populations were selected, having different levels of expertise: (1) students in geosciences, (2) researchers in geosciences and (3) confirmed experts. These three populations evaluated the rockfall hazard level on the same site, considering two different methods: the Laboratoire des Ponts et Chaussées (LPC) method and a method partly based on the “slope mass rating” (SMR) method. To complement the analysis, the completion of an “a priori” assessment of the rockfall hazard was requested of each population, without using any method. The LPC method is the most widely used method in France for official hazard mapping. It combines two main indicators: the predisposition to instability and the expected magnitude. Reversely, the SMR method was used as an ad hoc quantitative method to investigate the effect of quantification within a method. These procedures were applied on a test site divided into three different sectors.

A statistical treatment of the results (descriptive statistical analysis, chi-square independent test and ANOVA) shows that there is a significant influence of the method used on the rockfall hazard assessment, whatever the sector. However, there is a non-significant influence of the level of expertise of the population the sectors 2 and 3. On sector 1, there is a significant influence of the level of expertise, explained by the importance of the temporal probability assessment in the rockfall hazard assessment process. The SMR-based method seems highly sensitive to the “site activity” indicator and exhibits an important dispersion in its results. However, the results are more similar with the LPC qualitative method, even in the case of sector 1.

Rockfall instabilities are a major hazard for people, human activities and
infrastructure (Bell and Glade, 2004; Moreiras, 2006). It is thus essential
to assess the rockfall hazard in areas over which they are likely to occur
and to propose a mapping of the hazard to manage the risk in urban areas.
Natural hazard zoning has been introduced in many countries all over the
world. For instance, the PPR (Plans de Prevention des Risques Naturels
Previsibles) in France (Besson et al., 1999) and the

Rockfall hazard can be defined as the probability that a specific location at the toe of a studied slope will be reached by a rockfall of a given magnitude (Jaboyedoff et al., 2001). Whatever the magnitude of the expected rockfall, the probability can be divided into two terms: the failure probability and the propagation probability (Jaboyedoff et al., 2005).

Various methods simulate the trajectory of rock masses fragments after rupture and evaluate the propagation probability (Dorren, 2003). However, to date, there is no fully reliable method to estimate the failure probability (Hantz, 2007). Even though some existing methods are quantitative and based on historical inventory (Dussauge-Peisser et al., 2002; Hungr et al., 1999; Luckman, 1976), in most cases such an inventory is not available. Then, qualitative assessment methods are used to assess the rockfall probability failure. These methods are mainly based on expert judgment (Abella and Van Westen, 2008; Budetta, 2004; Effendiantz et al., 2004; Hantz et al., 2003; Jaboyedoff et al., 2001). Therefore, experts in charge of the hazard assessment have a key role. The method considered could likewise have an influence in the hazard assessment. Different research works (Abbruzzese and Labiouse, 2010; Bormioli et al., 2011) have compared different mapping methods and evaluated the influence of the method chosen on the obtained results. These studies highlighted the statistical significance of the chosen method: the use of one method over another gives different levels of failure probability for the same sector.

In this paper, an experiment is undertaken in order to evaluate the influence of the method and the level of expertise on the rockfall failure probability assessment. To simplify the reading, and the understanding of the paper, the term “rockfall hazard” will be used when referring to the rockfall failure probability. It corresponds to the combination of the occurrence probability, the temporal probability and the magnitude (volume). The rockfall hazard is assessed on a test site by three populations with different levels of expertise. Two different methods for rockfall hazard assessment are used: (i) a qualitative one, the Laboratoire des Ponts et Chaussées (LPC) method, which is the method mainly used in France, and (ii) a quantitative one, based on the slope mass rating (SMR). The experiment was conducted in two phases: the first one was realized in May 2012 and the preliminary results were presented in EUROCK Congress (Delonca et al., 2013). The second one was realized in May 2013, in order to confirm the first results, by increasing the size of the three populations and adding statistical procedure to the study. For each phase, the same protocol has been followed: three populations with different levels of expertise have assessed the level of rockfall hazard by considering two different methods of analysis, on three sectors of the same test site. The experimental protocol considered in the study is presented in more details in the first part of the paper. Then, the results are presented and discussed for the two phases together.

The objective of the study is to consider different methods of rockfall hazard assessment and to evaluate the differences in terms of levels of rockfall hazard. The levels of rockfall hazard considered in the study correspond to classical rockfall hazard levels: very low, low, moderate and high (Copons et al, 2008; Bauer, 2011; OFAT, 1997). These levels are commonly used to build hazard maps for risk management in urban areas. Thus, it is possible to compare these levels, which are obtained using different methods. Moreover, the influence of the level of expertise on the result, and so on the obtained hazard levels, is also investigated. Indeed, it may interest engineers and researchers in charge of hazard mapping or those concerned with the development of rockfall assessment methods. Note that the objective of the paper is not to evaluate the “true” level of rockfall hazard but to compare the evaluation process, considering different levels of expertise and different methods.

The experiment was realized on a cliff situated in Liverdun, a town close to Nancy in France. The test site has been chosen because of its history. It is an old quarry that became a climbing site, which has been closed after repetitive rockfalls. Moreover, it has been previously classified as presenting a high-level rockfall hazard (Moiriat et al., 2008). The methodology used to evaluate the rockfall hazard on the site is the following: (1) bibliographic review of all the documents available and (2) field recognitions. The first step leads to evaluate the risk area at a departmental scale. The second step leads to prioritize the level of rockfall hazard on the area of study. Therefore, the evaluation is made at a regional scale. The site is a 50 m long cliff of Jurassic limestone that is made up of massive blocks at the base and small blocks in the upper part (Delonca et al., 2013).

Three sectors were identified (Delonca et al., 2013):

Sector 1 presents a massive wall with well-defined stratification beds. A pluri-decametric fracture isolates a rock panel several hundreds of cubic meters in size. At the top, small unstable blocks are present (Fig. 1a).

In sector 2, two major fractures form a rock wedge of a few cubic meters in size. Small unstable blocks are present in the upper part (Fig. 1b).

In sector 3, limestone beds are overhanging, and some blocks at the top are unstable (Fig. 2).

Sector 1

Three populations were involved in the experiment.

A total of 38 first-year MSc students in the geosciences, confronted with a hazard assessment study for the first time. These students may later be asked to prepare hazard or risk maps, for example during an internship.

A total of 10 researchers in the geosciences, working in the field of hazard and risk assessment but not accustomed to regular rockfall assessment studies.

A total of 8 confirmed experts benefiting from a long experience in rockfall hazard assessment and risk studies.

Sector 3 and its main characteristics (Delonca et al., 2013).

Quantification of LPC parameters.

At first, an a priori assessment was requested from each population, without making use of any method. Then, two rockfall hazard assessment methods are used: the LPC method, which is a qualitative method frequently used in France, and a SMR-based method that relies on more quantitative information.

The first assessment of the rockfall hazard was requested from each individual of the three populations. Four failure probability levels were proposed: (i) zero, (ii) low, (iii) medium and (iv) high. A presentation of rockfall hazard theory and a presentation of the methods and their parameters were delivered before this work as well as a document containing additional information on the history of the site. The objective of this first assessment was to compare it with the assessment later carried out using the two other methods and to estimate the differences between an a priori assessment and one based on the use of a guided method.

The LPC method is detailed in Laboratoire des Ponts et Chaussées (Effendiantz et al., 2004). It is a qualitative method that is frequently used in France for official hazard mapping. It consists of two main steps: (1) the collection of preliminary data and (2) the use of these data to localize and characterize the potential instability.

The collected preliminary data incorporate all relevant information regarding the study site, including the following.

Documentary information: the objective is to exploit the archives to avoid duplication of studies already realized.

Historical information: to study the past of the site to determine whether events have already occurred on the site.

Geological information: to analyze the lithological and stratigraphic data and the regional geological history to establish the geological context of the study site.

Structural information: to identify the structural characteristics of the study site on different scales and to propose global and local structural models.

Morphological information: to identify the main historical steps leading to the actual morphology.

Hydrogeological, hydrological and climatic information: to characterize the fluid intakes, their nature and their importance. It is also to identify the flows inside the massif and unfavorable climate patterns (e.g., freezing, thawing and important thermal contrasts).

Information regarding vegetation: to identify and characterize the main vegetation on the site and its influence on stability or instability processes.

Potential mechanism of rupture: to identify the potential mechanisms of rupture associated with the studied zone.

Sectoring: to identify homogeneous areas from the hazard characterization point of view. This step is performed in the study of large zones.

Localize and identify potential instabilities. In this stage, the expert in charge of the study observes the following parameters:

Nature of the studied area: description of the instable rock mass, lithology, hydraulic and hydrologic data.

Geometric parameters: geometry and dimensions of the potentially unstable compartments.

Geomechanical parameters related to discontinuities: e.g., spacing, roughness, apertures, filling and orientation of critical discontinuities.

Triggering factors: e.g., interstitial water, rainfall, high temperature variation, freeze-thaw cycle and vegetation

Qualitative scale of the level of occurrence probability, after the guide of the Laboratoire des Ponts et Chaussées (2004).

Qualitative scale of the level of temporal probability, after the guide of the Laboratoire des Ponts et Chaussées (2004).

Matrix used to assess the coupled “temporal probability/occurrence probability”, after the guide of the Laboratoire des Ponts et Chaussées (2004).

Qualitative scale of the predisposition to instability.

This stage helps to determine the unstable volume (magnitude) and the potential mechanism of failure. In the experiment proposed in this paper, these parameters were coded to process them statistically (Fig. 3). This codification, which has been validated by expert users of the LPC method for the purposes of this experiment, does not belong to the original method.

Define the rockfall hazard. The analysis of the
previous parameters helps to define potentially unstable volumes and the
potential mechanism of rupture associated with these volumes. For each of
these, a coupled “temporal probability / occurrence probability” is assessed.
These two terms are qualitatively defined as follows.

The occurrence probability is subjectively assessed from the parameters presented in Fig. 3. It answers the question, “Can the rockfall occur?” The qualitative scale of the level of occurrence probability is presented in Table 1.

The temporal probability corresponds to the annual frequency or return period of the rockfall on the study site. It is defined as the time expected to failure. In other words, the temporal probability is defined by the probability that the failure occurs before an expected delay. The periods are defined on a scale from “imminent” to “long term”. The definitions of the terms are presented in Table 2.

Qualification of the probability of propagation. With the LPC method, it is proposed to carry out a qualitative trajectory analysis before considering a more advanced study using numerical simulations. We do not detail this last step because it has not been considered in the experiment.

The SMR index, proposed by Romana (1985), is a
geomechanical classification commonly used for the characterization of rock
slopes (Corominas Dulcet and Mavrouli, 2009; Irigaray et al., 2003) and
derived from the rock mass rating (RMR) as follows:

The strength of the intact rock.

The rock quality designation (RQD) (Deere and Miller, 1966), which gives a quantitative estimate of the rock mass fracturing based on the study of cores obtained by drilling. The RQD is defined as the percentage of intact pieces of length greater than 10 cm over the total length of the hole. It can also be estimated from surface measurements.

The spacing of discontinuities.

The condition of discontinuities: the roughness, weathering and opening of the discontinuities are assessed.

The water inflow through discontinuities and/or the pore pressure ratio.

Assessment of the rockfall hazard with the LPC method.

L: low; M: moderate; H: high

Assessment of the predisposition to instability with the SMR method.

VF: very favorable; F: favorable; M: moderate; U: unfavorable; VU: very unfavorable; VL: very low; L: low; M: moderate; H: high.

Assessment of the rockfall hazard with the SMR method.

VL: very low; L: low; M: moderate; H: high.

Mean and standard deviation of the rockfall hazard levels for the three sectors.

The SMR calculation leads to five stability classes (Romana, 1985). Then,
based on the work of El-Shayeb et al. (1997) or El-Shayeb (1999), a level
of the site activity is evaluated. There are four different levels of
activity corresponding to the following.

Sleeping: weathering traces are faded, and there is no alteration of the rock mass.

Inactive: few weathering traces and superficial alteration.

Low active: recent weathering traces and deeper rock mass alteration.

Active: numerous weathering traces and deep rock mass alteration.

Note that the level of activity, assessed considering the weathering traces is considered as the history of the site (El-Shayeb et al, 1997). Thus, it includes the temporal activity of rockfalls. Indeed, it is considered that some weathering traces can be used to evaluate the imminent and very short-term qualitative scale of the level of temporal probability (Table 2 – LPC method). Thus, even though this approach is less global than the one proposed by the LPC, it allows the rockfall hazard level to be evaluated. The different terms used in both the LPC and SMR-based methods are presented in Fig. 4.

The on-site observations took place at the foot of the cliff. Each individual had a compass, a sclerometer and a geologist's hammer with him and had to report his observations and measurements on a separate form for each sector. On this form, each method was presented again, and all the used parameters were listed (cf. Fig. 3 for the LPC method).

To statistically treat the results, the level of the rockfall hazard was coded as follows.

Low-rockfall hazard level: code value equal to 1.

Moderate-rockfall hazard level: code value equal to 2.

High-rockfall hazard level: code value equal to 3.

Terms considered in the LPC and SMR-based method.

Each individual had to make three types of rockfall hazard assessment: an a priori assessment, an assessment with the LPC method and an assessment with the SMR-based method. After the experiments were carried out, a descriptive statistical analysis was performed to compare the levels of the rockfall hazard assessment. Then, the results were analyzed using the chi-square independence test and the analysis of variance (ANOVA) method (Scheffé, 1959) to assess the significance of the influence of the level of expertise and of the influence of the method used in determining the rockfall hazard level. Finally, the means of the LPC and SMR-based method parameters were compared to identify those that most influence the rockfall hazard assessment.

Rockfall hazard level for

An initial statistical treatment has been performed for the three sectors. Figure 5 and Table 8 show the results and exhibit the following.

Regarding the a priori assessment, there is a wide dispersion (standard deviation higher than 0.5) of the results in terms of rockfall hazard levels for all populations in all sectors, except for the experts in sector 3. Except for sector 3, where the populations give similar results, the students more often give a higher value for the rockfall hazard level, followed by the experts and finally the researchers. This can be explained by the level of inexperience of the students, who were initially impressed by the presence of cracks and overhanging blocks on the cliff.

Regarding the LPC method, the results are similar for all population
groups, with less dispersion than with the a priori assessment. Students
achieved the highest mean score whatever the sector. Experts and researchers
are closer to each other, particularly in sectors 2 and 3. For comparison
purposes, a random sampling (using uniform distribution) of the levels of
predisposition to instability and magnitude provided a 6 / 25

With the SMR-based method, the results exhibit more dispersion for
the three sectors, with the standard deviations being higher than 0.5 except
for the students in sector 2. We also note that the mean rockfall hazard
level is always lower than that from the LPC method. For comparison, a random
sampling on the SMR matrix (predisposition to instability – magnitude) gives
the following results (number of favorable cases on number of possible
cases): a 6 / 16

The influence of the level of expertise and of the chosen method on the level of rockfall hazard could not be clearly determined from this preliminary descriptive analysis. This is the reason why statistical tests have also been performed.

The chi-square independence test allows the dependence between two qualitative variables to be investigated.

Let

Under the hypothesis that variables

Contingency table of the observed values of the

Contingency table of the observed values for the chi-square test performed between the level of expertise and the level of rockfall hazard (sector 1).

This test is performed side by side, between (i) the method and the level of
rockfall hazard (here considered as a categorical variable) and (ii) the
level of expertise and the level of rockfall hazard. As an example, Tables 10
and 11 present the contingency tables of the observed (

The

This table highlights the following:

a non-dependency between the level of expertise and the level of rockfall hazard for sectors 2 and 3;

a dependency between the level of expertise and the level of rockfall hazard for sector 1;

a dependency between the method and the level of rockfall hazard for the three sectors.

ANOVA is a statistical test indicating the influence of qualitative variable(s) on a quantitative variable to be assessed. It is based on the comparison of the mean values of the quantitative variable for each category of the qualitative variable.

Before performing the ANOVA, the normality of the level of rockfall hazard (the quantitative variable) and its homoscedasticity have to be verified. The levels of rockfall hazard are discrete data, which make the condition of normality difficult to assume. However, authors such as Legendre and Borcard (2008) and Underwood (1996) state that, in a balanced experimental design, ANOVA has problems with the heterogeneity of the variance when the variance is in high contrast to others, but the ANOVA is little affected by non-normality. This condition is considered as not discriminating to achieve the ANOVA.

To test the homogeneity of the variances, there are several tests, including Levene's test, that are insensitive to the normality of the data (Legendre and Borcard, 2008). If Levene's test is statistically significant, the assumption of the homogeneity of variances should be rejected. We used the software program R to perform this test. For the three sectors, Levene's test has shown that we can consider the variances to be homogeneous.

Contingency table of the theoretical values for the chi-square test performed between the level of expertise and the level of rockfall hazard (sector 1).

Chi-square distance between the observed and theoretical values for all the cases tested, under the hypothesis of independency.

Results of ANOVA: influence of the level of expertise and the
choice of the method on the level of rockfall hazard for the three sectors.
The bold values highlight the significantly influencing factors, for which
the

Code of the parameters used in the LPC and SMR-based methods.

In the ANOVA, the influence of a factor is considered significant when the

a very significant influence of the factor “method” on the level of rockfall hazard assessments for the three sectors;

a very significant influence of the factor “expertise” on the level of rockfall hazard assessments for sector 1;

the non-significant influence of the factor “expertise” on the level of rockfall hazard assessments for sectors 2 and 3 at a 1 % risk (but a significant influence at a 5 % risk);

the absence of interaction between these two factors on the level of rockfall hazard assessments (for all three sectors) at a 1 % risk or even at 10 % risk.

To better understand the influence of the LPC and SMR-based methods on the level of rockfall hazard, a comparison of the mean values of the parameters used in these methods has been conducted (Table 14).

For each parameter presented, a two-by-two comparison of the mean values has been carried out. More precisely, we have compared the mean values of the levels given for each parameter by (i) the students and the researchers, (ii) the students and the experts and (iii) the researchers and the experts. The objective was to identify the mean values that are statistically similar and those that are different. This analysis could explain the influence of the chosen method previously highlighted, depending on the sector and the level of expertise.

The first step consists of checking the hypothesis of equal variance for each
couple of the population. If the variances can be considered equals, it is
then possible to compare the mean values. Let

Let

Table 16 summarizes the results and shows the following:

the predisposition to the instability (LPC method only) is the same, whatever the level of expertise and the sector;

the magnitude is similarly assessed for sectors 2 and 3, regardless of the level of expertise, for both methods (LPC and SMR-based methods);

the magnitude is assessed differently by the students on one side and the experts on the other side for sector 1;

The SMR classes in the SMR-based method is the same for sectors 2 and 3, but not for sector 1, where there is a significant difference between the students and the experts;

the level of activity is assessed differently for the three sectors. Then, seeing that the combination of the SMR classes and the site activity provides the level of predisposition to the instability, the predisposition to the instability is assessed differently for the three sectors.

Note that another analysis (principal component analysis) has been realized to compare directly the parameters used to evaluate the occurrence probability of the LPC method (Fig. 3). However, the analysis did not produce any concluding results.

Values of the

Sample mean and standard deviation of the values of the LPC and SMR-based parameters. The bold values correspond to the population means that cannot be significantly differentiated.

During the experiment, three methods were used to assess the rockfall hazard. First, each participant was asked to assess the level of rockfall hazard from his own experience without using any tool, form or measurement. Second, a qualitative method (LPC method) and, third, a quantitative method (SMR-based method) were used. In both of these latter cases, a form was provided to guide the process of assessment.

We chose to use the LPC method due to its wide use in France for most official
hazard mapping. This methodology aims to formalize the practice gradually
developed over 2 decades through many field studies produced by both
operational services of the French government and local authorities. Many
experts contributed to the preparation of this guide, and some of these
experts also participated in our experiment. The methodology proposed by the
LPC method is somehow classical and follows the following steps as
previously described:

a literature review, which avoids repeating studies or investigations already done;

an historical review, which makes an initial zoning of hazard levels, taking into account past events;

a geomorphological analysis, during which the level of the rockfall hazard is actually assessed.

Then, even though the LPC method is not an internationally recognized method, we may consider that it treats the hazard analysis in a common manner, similar to other methods. In contrast, the SMR method, also used in the present work, is commonly used in rock slope stability analysis along roadways (Budetta, 2004; Corominas Dulcet and Mavrouli, 2009; Irigaray et al., 2003; Tomas et al., 2012). It is derived from the globally used RMR method for the stability study of excavated slopes or underground excavations.

Even though it is a qualitative method, the LPC method helps the user to identify the block masses that are potentially instable and the volume concerned and leads to the assessment of the level of rockfall hazard in a similar manner by the three tested populations. However, the introduction of a paper form to assist in the process of the assessment of the level of rockfall hazard could have introduced a bias favoring the LPC method in terms of less dispersion in the assessment results when compared with the results of the SMR-based method. As a result, such a form could be promoted to engineers making use of the LPC method to enforce a common interpretation of some aspects of the method.

The SMR-based method shows results similar to those obtained by the LPC method for students. However, for researchers and experts, the dispersion in the results is more important. The assessment of the site activity is not precise enough to allow the same level of activity to be assessed by all the individuals. Surprisingly, the use of a partly quantitative method does not prevent differences in the final rockfall hazard level, mainly because of the assessment of the site activity, which is a subjective component of the method. The terms used to describe the activity may be clarified to reduce this effect. Moreover, the prevailing failure mechanism on the three sectors plays an important role in the use of the SMR-based method. Indeed, the SMR method is routinely used to assess rockfall susceptibility of failure mechanisms involving shear on joints (including flexural toppling) though it seems inapplicable to failures caused by traction. In the test site, the main failure mechanisms are column toppling, wedge slide and overhang failure. However, there also exists a failure mechanism that involves shearing of joints on the three studied sectors. Because it is not the main failure mechanism on the test site, the SMR-based method has been considered anyway into the analysis.

For the experiment, one test site was selected, among which three sectors could be defined. These three sectors provide repeatability in the analysis and produce more relevant results. However, because there are three sectors on the same site, all of them are composed of the same lithology. Hence, the results obtained in the present study only apply to this case study, and the experiment should be repeated on other sites with different lithology and geological structures to prove that the findings are generic. The test site was chosen because of its history, as it was a climbing site which has been closed after repetitive rockfalls. However, no quantitative information concerning the occurrence of the phenomenon was available. The site is not monitored, and the sole data considered in the analysis were the visual evidence of events. The scenario considered here is that an expert is in charge of the evaluation of a new site, and he does not have any temporal data available to him. Therefore, to propose a more complete analysis, it would be interesting to consider another test site where the estimation of temporal frequency is possible. A national French project is underway that may provide a framework to repeat this experiment on different study sites, with more students, researchers and experts involved, as well as other methods to be tested. However, no sites have yet been chosen to carry out this experiment again.

As explained previously, to carry out the statistical analysis, the three
rockfall hazard levels were transformed into quantitative variables: the
low-rockfall hazard level was coded to 1, the moderate-rockfall hazard
level to 2 and the high-rockfall hazard level to 3, so as to reflect the
increase of the rockfall hazard in a familiar and convenient way. However, to
confirm that this coding has no influence on the results, we performed
additional ANOVAs with different encodings. The following cases were used,
respectively, for low-, moderate- and high-rockfall hazard levels:

0, 5 and 10;

1, 2 and 5;

1, 4 and 5;

3, 2 and 1 (converse coding).

Note that the quantification of the variable is used only for the statistical analysis.

The novel experiment developed in this paper had the goal of statistically evaluating the influence of the level of expertise and the choice of the method used for the rockfall hazard assessment. Three levels of expertise (students, researchers and experts) and two methods (LPC and SMR-based, plus an a priori assessment) were used on three sectors at one test site.

The main result obtained is that the influence of the level of expertise is not significant on the rockfall hazard assessment, which means that geoscience students and geoscience researchers who are not experienced at rockfall evaluations provide the same assessment as engineers experienced in rockfall hazard assessment.

More precisely, the qualitative analysis of the results, as well as the chi-square independence test and the ANOVA test highlighted the following:

a statistically significant influence of the factor “method” on the level of the rockfall hazard assessment for all three sectors;

a statistically significant influence of the factor “expertise” on the level of the rockfall hazard assessments for sector 1;

a statistically non-significant influence of the factor “expertise” on the level of the rockfall hazard assessments for sectors 2 and 3.

Moreover, the statistical analysis showed a high dispersion in the rockfall hazard assessment for the a priori assessment, especially for the students and researchers, explained by their inexperience in this type of exercise. This result, coupled with the previous one, highlights the “plus-value” of the experts: they do not overestimate the level of rockfall hazard in complex cases, and their assessment is more homogeneous and based on similar cases they have already studied due to their experience. Indeed, for sector 1, which exhibits the above mentioned pluri-decametric fracture, the results show a higher ratio of low-rockfall hazard levels for the experts. This suggests that the experts link the rockfall hazard assessment to the temporal probability assessment. For them, there is no risk of failure within a short time.

Other assessments of the rockfall hazard level will be realized on other sites to enrich this experiment and confirm or challenge the conclusions. The rockfall hazard level will be assessed by other people (students, researchers and experts), and other methods will be used.

If confirmed, the conclusions drawn in the presented paper would suggest that the use of a qualitative approach could be as relevant as a quantitative approach for rockfall hazard assessment even though it leaves room for some subjectivity. Nevertheless, the use of a standard form as proposed in this study (as exists for many other methods, either basically quantitative or qualitative) could be promoted to engineers making use of the LPC method to enforce a common interpretation of some aspects of the method. Such a form may contribute to a reduction of the subjectivity expected in a qualitative method.

The authors are indebted to all the people (students, researchers and experts) who participated in the experiment and the region of Lorraine for its financial support.Edited by: T. Glade Reviewed by: two anonymous referees