the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 15 Jan 2024
Testing the 2020 European Seismic Hazard Model (ESHM20) against observations from Romania
Abstract. Evaluating the performance of probabilistic seismic hazard models against recorded data and their potential to forecast future earthquake’s ground shaking is an emerging research topic. In this study, we evaluate and test the results of the recently released European Seismic Hazard Model (ESHM20; Danciu et al., 2021) against observations for several cities in Romania. The dataset consists of ground shaking recordings and macroseismic observations, which extend the observational time-period to few hundred years. The full distribution of the hazard curves, depicting the epistemic uncertainties of the hazard at the given location was considered and the testing was done for peak ground acceleration (PGA) values i.e., 0.1 g and 0.2 g.
The results show close agreement between the ESHM20 and the observations for the cities located near the Vrancea intermediate-depth source (VRI) for both selected PGA levels. ESHM20 appears to overestimate the VRI recorded ground motions along the Carpathian Mountain Range and underestimate those at the far-field locations outside the Carpathians. Some of these differences might be attributed to the uncertainties in data conversion, local site effects, or differences in the attenuation patterns of the ground motion models. Our analysis suggests that the observed exceedance rates for the selected PGA levels are consistent with ESHM20 estimates, but these results must be interpreted with caution given the limited time and spatial coverage of the observations.
- Preprint
(1405 KB) - Metadata XML
-
Supplement
(576 KB) - BibTeX
- EndNote
Status: final response (author comments only)
-
RC1: 'Comment on nhess-2023-232', Graeme Weatherill, 31 Jan 2024
Summary
The manuscript presents the results of a quantitative comparison of the observed rate of exceedance of given levels of ground motion (0.1 and 0.2 g PGA) for 12 cities across Romania against the distribution of expected numbers of exceedances according to the full logic tree of the 2020 European Seismic Hazard Model (ESHM20). The comparisons themselves are based on macroseismic intensity data, which, although uncertain measures of observation when compared against predictions of engineering ground motion parameters such as PGA, provide a longer history of data for the sites in question. The comparisons show that while the uncertainties are considerable, and the volume of available data still quite limited, the predicted rates of exceedance from ESHM20 are not inconsistent with those observed for the cities shown.
Quantitative comparisons between probabilistic seismic hazard models and observations (even indirect ones from intensity) are often highly uncertain, meaning that only limited conclusions can be drawn. Nevertheless, when executed carefully and with these uncertainties taken into consideration, they represent a first order check that can help us understand whether the models themselves are consistent with reality. The study presented here provides a useful comparison and is presented by the authors in such a way that the inconsistencies and limitations are clearly acknowledged and, where possible, addressed. The paper does not overstate the strengths of the conclusions reached and the authors attempt a reasoned interpretation of the city-by-city differences in the comparisons, which although not conclusive could assist in improving future versions of the model. The manuscript is generally well written and organized; however, there are certain steps of the methodology, particularly those relating to the handling of uncertainties in the observed exceedances and the statistical models used for the comparison, that are not sufficiently well described.
I believe that the paper constitutes a valid and worthwhile contribution to this special issue of Natural Hazards and Earth System Sciences on the topic of Harmonized Seismic Hazard and Risk Assessment for Europe. I am therefore happy to recommend this paper for publication in the special issue subject to moderate revision, and ask the authors to address the comments listed below.
General Comments on the Manuscript
- Figure 1A is a somewhat poor quality map that lacks sufficient information to serve as a general reference map to indicate the location of the cities, the hazard and/or the notable geographic regions referred to in the text. I would recommend separating Figure 1 in two separate figures, and perhaps prepare two different maps. One containing the main geographical features to serve as a general reference and the other showing the relevant seismological and seismic hazard features. If the authors prefer to keep one map, then this should be enlarged and with more of the relevant geographical regions, such as the Carpathian Mountain belt, clearly labelled.
- The authors indicate that to undertake the comparisons they selected only intensity data points (IDPs) from events with Mw ≥ 6 for the Vrancea zone and Mw ≥ 5.0 for shallow seismicity. However, the minimum magnitude for both regions considered in the ESHM20 model is Mw 4.5, meaning there is some inconsistency in the rate of occurrence of earthquakes implied in the model and those selected for the comparison. Could the authors expand more on the rationale and the implications of this selection for the comparisons?
- On page 5 (lines 144 – 147) the authors indicate that they “correct for site effects considering amplification in each city by means of soil factors recommended in Eurocode 8”. There is quite an important step (and assumption) here that is not being addressed in significant detail. From this description I would interpret that the authors are taking a single Eurocode 8 site class for a city and based on that site class simply scaling down the PGA to the equivalent rock PGA by dividing it by the EC8 amplification factor. If this is the case, then I would ask the authors to expand on this a bit more in order to clarify from where they obtain the site class adopted for each city, whether this is based on measurements/microzonations or on inference from a proxy, over what spatial area is the class representing (i.e. how homogeneous might the site classes be for each city) etc. This may be relevant as the authors are mostly limiting their data to larger IDPs, which may represent stronger shaking. If the cities are located on soft or deep soils then there may be a greater degree of soil nonlinearity influencing the results than implied by the previous Eurocode amplification factors (CEN, 2004). The appropriate handling of site response in this comparisons is a complicated topic and while there may not be a clear best practice for addressing this issue, some awareness of the potential biases and uncertainties may help the reader interpret the results. In addition, I would recommend adding the assumed site class (and perhaps the data source) for each city into Table 1.
- In pages 5 – 8 it is mentioned in several places that the standard deviations of the intensity to ground shaking conversion models is taken into account (Lines 154 – 155, 167, 195 – 196) but it is not clear from the description of the process exactly how this is done. Please can the authors explain this in more detail.
- On page 7 (lines 175 – 183) a little more clarification is needed to understand what is meant by the weighted mean binomial distribution. My assumption is that this refers to a binomial distribution in which the probability parameter is determined as the mean of the probabilities of exceedance for the given acceleration level given by each branch of the logic weighted by the respective branch weights (which is more or less what I understand is the process described in the Stirling et al., 2023 paper). But then in the supplementary materials the authors compare probability distributions fit to the probabilities of exceedance of 0.1 g PGA for Bucharest (that doesn’t include a weighted binomial model), which would imply that the authors take a weighted mean as an expectation from a distribution fit to the PoEs. This seems to be a crucial step in understanding what the authors are doing but this paragraph (Lines 174 – 183) is really quite cryptic and not clear to the reader. Similarly, the step-by-step methodology on Page 8 (Lines 190 – 205) seems to skip over these details. I would strongly encourage the authors to revise these paragraphs (Lines 174 – 205) to provide a clearer description of the methodology, providing the appropriate formulae for the weighted binomial distribution and possibly removing the comparison of the different probability distributions in Figure S2 unless it is critical to developing an understanding of the methodology.
Editorial
Page1, Line 14: “i.e.” is not necessary.
Page 1, Line 24: “PSHA underlines a wide range of applications” – I would say “underpins” a wide range of applications if you mean that it serves as an input or foundation to them.
Page 1, Line 30: Gerstenberger et al. (2020) reference is missing in the bibliography.
Page 2, Line 35: “empirical data is limited to a short temporal window”. I think you are talking only about accelerometric data here, so I would suggest changing “empirical” to “instrumental records”, or something similar.
Page 2, Line 43 – 44: “with notable/maximum effects seen outside the epicentral area”. This is a strange phrasing so it’s not clear what is meant here. Are you trying to indicate that the regions of largest damage (or felt effects) were located away from the epicentre of the earthquake? I recommend a slight rewording here.
Page 2, Line 46: Marmureau et al. (2017) is not present in the bibliography. Do the authors mean Marmureau et al (2018)? If not, then please add the missing entry.
Page 2, Line 50: “The VRI impact overpass the national borders …”. “Overpass” is not the correct word to use here. I recommend simply saying that it “extends beyond the national borders …”.
Page 2, Line 50: “e.g., observed intensities” can be replaced with just “with observed intensities …”
Page 2, Lines 51 - 53: “while the shallow crustal seismic activity in Romania is not as frequent as the one at intermediate depths in the Vrancea region, it still poses a significant contribution to the regional seismic hazard”. A citation to support this statement would be beneficial here.
Page 2, Line 57: “…, indicating substantial effects on the affected regions”. This is an awkward tautology. The authors could perhaps add some more explicit detail rather than just “effects”, or they could simply remove this part of the sentence altogether.
Page 3, Lines 67 – 69: This last sentence of the paragraph can be removed as it is just stating the obvious and repeating the same references cited earlier in the introduction.
Page 3, Line 79: Clarify that in the ESHM20 seismicity from the Vrancea seismic region is modelled with a set of uniform area source zones located between about 70 to 150 km depth.
Page 3, Line 84: “a sampling technique of the full logic tree was used …” would be easier to just phrase that “the full logic tree was sampled …”
Page 4, Line 110: “The observations [of macroseismic intensity] were collected within this study and were not directly used in the derivation of any component of the ESHM20”. Perhaps the authors could clarify that this means the intensity data were not used as a basis to estimate the size and location of the historical earthquake catalogue that was used for ESHM20 (AHEAD, Rovida et al., 2020). Otherwise, if this is the same intensity data on which the AHEAD earthquakes were calibrated then it is not true to say that the data were not directly used in the derivation of any component of the ESHM20.
Page 4, Line 118: I recommend moving the reference in parenthesis to the end of the sentence.
Page 5, Line 133: “The testing dataset at the 12 major cities contains 199 IDPs recording from 58 earthquakes (…), from which 39 are located in the VRI region and are considered as main events in the ESHM20 declustered catalogue”. Are the other 19 earthquakes not located in the VRI region, or is it the case that they are not considered main events (or are they not present at all in the ESHM20 catalogue)?
Page 5, Line 140: The sentence needs an “and” between the Vacareanu and Marmureano references.
Page 5, Lines 142 – 144: “We decided to do the translation from IDP to PGA, as it is more efficient to convert the relatively small number of the reported intensities and more importantly, to minimize potential errors at the data levels, rather than at the results”. This sentence doesn’t seem clear to me. Do the authors mean that it is simply less challenging to convert the data from IDP to PGA than it is to convert the hazard curves from PGA to IDP? A little more clarification here would be useful.
Page 9, Lines 229 – 230: “The results at the cities … exhibit hazard predictions that reflect the frequent crustal activity as a significant attenuation behind the arc dampened VRI-related ground motion”. This sentence is not entirely clear to me. By crustal activity are you describing seismic activity or deformation, or both?
Page 10, Line 268: “source directivity effects which are significant for major events occurring in Vrancea”. A supporting reference citation is needed here.
Page 10, Line 269: “draw” should be “drawn”.
Page 13, Table 1: The quantities in the table don’t seem to be entirely clear and are difficult to reconcile with the explanation in the caption. Specifically, the caption indicates that “PE” refers to the “ESHM P-value”, and “PG” the “ESHM annual probability of exceedance”. Presumably the PE value here refers to the P of the weighted binomial distribution used in the comparisons, but the values for PG don’t make sense. Firstly, it is not possible that the annual probability of exceedance of 0.2 g is greater than that of 0.1 g. Secondly, for 0.2 g the annual probabilities of exceedance equal 1.0, which doesn’t correspond to any value in the ESHM20 model. Perhaps these values refer to the probability of exceedance in the time period observed for the station (still high in some cases), in which case the mistake is in using the word annual here. The authors should re-check the values they are entering in the table and where necessary revise the caption accordingly.
Page 14, Line 324: Please indicate which version of the OpenQuake-engine was used for the analysis (in addition to the date of access). This is important as calculation details can change between versions, which affects reproducibility.
Citation: https://doi.org/10.5194/nhess-2023-232-RC1 -
RC2: 'Comment on nhess-2023-232', Céline Beauval, 29 Feb 2024
Review of « Testing the 2020 European Seismic Hazard Model (ESHM20) against observations in Romania”
By Elena Manea et al.
The manuscript is overall a well-written and clear document, presenting a sound work that aims at comparing ground-motion occurrences, as forecasted by the ESHM20 model, with available observed data (macroseismic intensities and ground motions).
The overall methodology is sound. Uncertainties on the different steps are accounted for. Besides, the authors acknowledge the high level of uncertainties related to such comparison, the limits associated to the length of observation time windows, and they provide fair conclusions.
There are a few issues that would need to be addressed to strenghten the manuscript, easy to implement, that I list below:
- Geographical area for Carpathian? This geographical mountainous zone is mentioned many times throughout the manuscript, it should be delineated on Figure 1, for those who are not familiar with the region. The map in Figure 1 could be larger and one figure on its own (relief is difficult to distinguish).
- I might be missing something, but the hazard levels indicated in the text do not fit the levels that can be inferred from the hazard curves in Figure 1B ? at return period 475 years (or 0.0021 annual probability of exceedance, for example in Focsani a PGA larger than 1.0g is obtained (line 91, it is written 0.75g?) ?
- The reader is not given any info on how the completeness was evaluated ? The assumption that all observations above a given level are reported over a given time window is essential to the methodology applied.
- There is a confusion around the ‘time period’ mentioned in Section 4. When you write ‘we determine the specific time period of this dataset and count the instances ..’, do you mean you determine the complete time windows for a given ground-motion threshold ? This confusion is also present in the explanation of the different steps of the methodology (page 8). Item 1 states ‘estimate the time-period of available ground motion for each city’, do you mean the time period of completeness for a given ground motion level. Only item 5 mentions that the time window it the time window of completeness (for the considered ground-motion level?).
- Can you please provide one sentence that explains the use of a binomial distribution ?
- Figure 3B is the figure explaining how distributions for numbers of exceedance are estimated, but it lacks information. The caption does not state that it is the number of exceedances of the ground-motion level 0.1g, over a given time window ? which is 322 years for Bucharest ? A vertical bar at 0.1 would be useful on Fig. 3A. Figure S2 is important to understand what is done, it would be nice to display it in the manuscript rather than in the Electronic supplement.
- When discussing the differences between the forecast and the observation, only issues related to the ground-motion model in the ESHM20 model is addressed. Why not also questioning also the source model ?
- The authors are cautious when deriving conclusions for cities in low seismicity areas with few data. Nonetheless, it could be written more clearly that in these cities, there are too few data to really conclude anything, and that longer observation time windows would be required to highlight any difference (if there is any) between the forecast and the observation.
- Some paragraphs are very long, the reading would be easier if these paragraphs are broken into shorter ones.
More comments
Several time in the manuscript, the term ‘recurrence periods’ is used for earthquakes (e.g. line 28), to avoid confusion, keep periods or return periods for ground motions, and use e.g. recurrence times for earthquakes.
Line 96, in this case, it should be return period rather than recurrence period because it refers to ground motions.
Line 31: strong-motion network in France installed the mid 90s (not since the mid 70s)
Lines 35-36 : performed at short or long return periods
Writing of intensity level either ‘5’ or ‘V’, but this would need to be homogeneous throughout the manuscript
Line 110 ‘were not directly used in the derivation of any component of the ESHM20’ => this sentence would need to be softened, macroseismic intensities have been used in the building of the historical part of the earthquake catalog
Lines 128-134 : would it be possible to explain to the reader why the relation between PGA and intensity may be different for different classes of earthquakes (deep and shallow) ?
Line 133-134 : unclear, strong motion stations were already installed in the period pre-1977 in Romania ?
Please check these sentences that are partly obscure and would benefit from being rephrased to be better understood:
Lines 44-45 : ‘notable effects seen outside the epicentral area’, unclear, do you mean specific site effects have been observed ?
Line 178-179 ‘Note that the statistics are summarized for the Annual Probability of Exceedance (APE) and that the differences between the two statistical descriptors (..) is evident in Figure 4B’ : sentence that would need to be rephrased
Lines 220-221, increased number with respect to ? explanation provided is not that clear
Lines 247-248, ‘The observed consistency … for the 0.1g PGA level’ would need to be rephrased.
Lines 279-280 ‘we also acknowledge that the statistical testing are limited in scope given …’ : this sentence is not that clear and would deserve to be rephrased
Figures
Figure 1: caption of the figure should state how the earthquakes reported on the map have been selected (not all earthquakes post-1700 in the ESHM20 catalog are displayed?), explanation is only in the text
Figure 4 : need to state that these are numbers of exceedance over the time window of completeness
Citation: https://doi.org/10.5194/nhess-2023-232-RC2 -
RC3: 'Comment on nhess-2023-232', Anonymous Referee #3, 18 Mar 2024
The paper addresses an important step towards the acceptability of any seismic hazard model (in this case ESHM20), i.e., the compatibility of its outcomes with the ground shaking observations.
In particular, the paper describes the consistency of ESHM20 outcomes with the ground shaking observations in Romania.The paper is well written and clear. I have only minor suggestions except a couple of points (about the completeness of the datasets, and the misuse of terms like "underprediction" and "overprediction") that might require a more careful rewriting of some paragraphs.
Below I describe in detail my concerns.- LINES 84-85. The authors write "... a sampling technique of the full logic tree was used to obtain the distribution of the hazard results." I think that the authors should say something more about this kind of sampling, considering that the logic tree branches are (almost) never uncorrelated and sometimes they may not even randomly sample the epistemic uncertainty.
- In the selection of the macroseismic intensity data for the cities considered in this analysis, the authors never mention the problem of catalog incompleteness. I think that they should be clearer on this point since the data incompleteness may significantly affect the results of the comparison.
- LINES 142-144. The authors write "We decided to do the translation from IDP to PGA, as it is more efficient to convert the relatively small number of the reported intensities and more importantly, to minimises potential errors at the data levels, rather than at the results.". I do not understand the point relative to the "minimization" of potential errors on the data levels. I think that the authors should be clearer on this point.
- In many part of the section 5 and 6, the authors use the terms "underprediction" and "overprediction" inappropriately. In Figures 4 and 5 (and the ones in the supplementary material) the authors show that the ground shaking observations are inside the expected variability given by the forecasts. To me, this means that the outcomes of the model are consistent with observations. That is a good result.
To me (as I guess for many other readers) the terms underprediction/overprediction indicate a failure of the model. However, the authors use often these terms to merely indicate if the observations are below or above the expected mean number of exceedances.
In addition, the authors want to justify these underpredictions/overpredictions with the problems of the model (lines 265-270), whereas they could be entirely due to the sampling problem (the data represent only one realization of a stochastic model among many other possibles). In essence, if I have a fair coin and I toss it 10 times, the fact that I got 4 heads and 6 tails does not mean that my model (a fair coin that predicts 5 heads in 10 tosses) is overpredicting. Of course I may have missed some important authors' points here. In any case, I think that this point has to be deeply discussed to avoid the spreading of the misconception that a model (the mean hazard) has to predict exactly the observations. In my mind, if this happens, it is a clear evidence of an overfitting of the data, not that the model is good.- To my knowledge ESHM20 considers only "mainshocks", whereas the authors consider "all" ground shaking data for this comparison. The fact that the comparison between outcomes of the model and data is good seems to indicate that the data do not come from "clustered" earthquakes (or sequences). I cannot check that because I do not know the earthquake catalog for Romania used by the authors. In any case, if important sequences are part of the data I would have expected that ESHM20 may underpredict the observations. I think that a clearer discussion on this point is important.
- Figure S5 is very unclear to me. In particular, I do not understand how the annual frequency of exceedance (AFE) of the model (red line) is compared with the AFE obtained by the data. The equations that I know to estimate AFE and its uncertainty from data do not provide the black dashed lines shown in the Figure S5 and the 0 exceedances seem to be set arbitrarily to an AFE equal to 10^-4 (this has an impact also on the uncertainty that are associated to it). In essence, the dashed black lines shown in Figure S5 are not comprehensible to me, and they seem to be wrong. This figure has to be explained better, or even removed because it seems to me that is not adding any more information with respect to the figures 4, 5 and S3,S4.
- The authors conclude the paper writing "... but these results must be interpreted with caution given the limitations in the time and spatial coverage of the observations, both the ground shakings and the macroseismic intensity dataset. "
This is just a comment. I am ok with that but I do think that this is the same for all NSHM. We can merely check if the model is consistent with available data. Nothing more than that.Citation: https://doi.org/10.5194/nhess-2023-232-RC3
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 279 | 59 | 23 | 361 | 27 | 15 | 15 |
- HTML: 279
- PDF: 59
- XML: 23
- Total: 361
- Supplement: 27
- BibTeX: 15
- EndNote: 15
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1