Process-based modelling to evaluate simulated groundwater levels and frequencies in a Chalk catchment in south-western England

Brenner, Simon; Coxon, Gemma; Howden, Nicholas J. K.; Freer, Jim; Hartmann, Andreas

doi:https://doi.org/10.5194/nhess-18-445-2018

Articles | Volume 18, issue 2

https://doi.org/10.5194/nhess-18-445-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/nhess-18-445-2018

© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 18, issue 2

Research article

|

08 Feb 2018

Research article |

| 08 Feb 2018

Process-based modelling to evaluate simulated groundwater levels and frequencies in a Chalk catchment in south-western England

Simon Brenner, Gemma Coxon, Nicholas J. K. Howden, Jim Freer, and Andreas Hartmann

Download

Final revised paper (published on 08 Feb 2018)
Preprint (discussion started on 22 Dec 2016)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Referee comment', Andrew Long, 18 Jan 2017
- SC1: 'RC1 reply', Simon Brenner, 19 Mar 2017
RC2: 'Invited Review', Anonymous Referee #2, 16 Feb 2017
- SC2: 'RC2 reply', Simon Brenner, 19 Mar 2017
- AC2: 'Short comment (RC2 reply) again as author comment: "Reply to the review of referee #2"', Andreas Hartmann, 23 Mar 2017
AC1: 'Short comment (RC1 reply) again as author comment: "Reply to Dr Long's review"', Andreas Hartmann, 23 Mar 2017

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (further review by Editor and Referees) (06 Apr 2017) by Mario Parise

AR by Simon Brenner on behalf of the Authors (18 May 2017) Author's response Manuscript

ED: Referee Nomination & Report Request started (27 May 2017) by Mario Parise

RR by Anonymous Referee #2 (01 Jun 2017)

Suggestions for revision or reasons for rejection

Processed-based modelling to evaluate simulated groundwater levels and frequencies in a Chalk catchment in Southwest England.

Brenner at al.

This is a revised version of a manuscript detailing the simulation of groundwater levels in a Karst environment in the Southwest of England. I described in my review of the original manuscript how this topic is highly relevant to the journal and timely given the increasing need to simulate and forecast groundwater levels from limited datasets. The manuscript is suitably concise and the description is clear. I fully agree with the authors’ statement in their Abstract that “specialised modelling approaches are required that balance model complexity and data availability”. The authors assess whether they have achieved this balance by both exploring the identifiability of the parameters within their model (using a Shuffled Complex Evolution Method; SCEM) and by comparing model performance metrics for calibration and validation datasets (i.e. a split-sample test). They conclude that their modelling exercise had been a success because their analyses suggest that all of the parameters are identifiable and the differences between the calibration and validation metrics take values which they consider to be small.

Whilst I fully endorse the authors’ general approach to assessing the performance of their model I have severe concerns about the exact way in which it has been implemented. I do not believe that the posterior distributions of the parameters yielded from the SCEM accurately reflect the uncertainty of these parameters. Furthermore, I do not believe that the comparison between performance metrics upon calibration and validation are particularly meaningful. For these reasons I do not recommend that the manuscript is accepted for publication.

I first detail my concerns about the analyses of parameter identifiability. Looking at Figure 5, it is apparent that according to the SCEM that when the model is calibrated using only discharge data that the Kc parameter (for example) is almost perfectly identifiable. This posterior distribution indicates that this parameter definitely has a value less than 1. However, when all of the calibration data is used the parameter definitely has a value greater than 9. This is a clear contradiction and at least one of these two posterior distributions must be incorrect. Similar contradictions are evident for all of the other parameters except for those related to the groundwater level in a specific borehole.

Furthermore, the theoretical justification for the authors’ choice to use the Kling-Gupta efficiency (KGE) as the objective function within the SCEM is rather weak. The formal theory of Markov Chain Monte Carlo methods such as the SCEM require that the objective function is a likelihood (i.e. the probability that the data is realised from the proposed model). The authors indicate that the KGE can be treated as an ‘informal’ likelihood function and refer to a paper by Smith, Bevan and Tawn. This paper does discuss informal likelihood functions and describes sufficient conditions for informal likelihood functions to satisfy the most fundamental axioms of a probability. As far as I can see, the paper does not explicitly mention the KGE. The starting point for satisfying the axioms of probability is that the informal likelihood function can be written as an Lp-norm. It is not immediately clear to me that the KGE can be written as an Lp-norm. Therefore, I am unclear of the relevance of the Smith et al. paper to the authors’ study and I am not convinced that the KGE satisfies the fundamental axioms of a probability. I would have thought these axioms were a necessary requirement for a function to be treated as a likelihood.

The authors do not provide any calibration diagnostics which might indicate that the SCEM has converged to a stable posterior distribution.

The authors do not conduct any validation tests which might indicate that the posterior distributions reflect the uncertainty of the model parameters.

I similarly have a number of concerns about the authors split sample tests. First, the authors conclude that decreases in model performance upon validation of 11 and 21% are sufficiently small to indicate robust model performance. They refer to other studies where a similar decrease in performance was observed. However, the decision that 21% is ‘sufficiently small’ is completely subjective. The expected decrease will be a complex function of the number of observations and the degree of seasonality, variability and autocorrelation realised by the data. Therefore, the comparison with other studies is irrelevant. This being said, if I were to compare these values with the results of modelling exercises I have previously undertaken then I would consider 21% to be a relatively large decrease.

In their ‘Responses to comments’ document the authors describe how the KGE objective function “was chosen by trial and error comparing the simulation performances during calibration and validation obtained different objective functions (RMSE and other)”. This ‘trial and error’ approach very much concerns me. The validation data should not be involved with the model calibration in any way – this includes the decisions about how the model is calibrated and what objective function is used. In my opinion, the use of the validation data in this manner invalidates the authors’ split-sample tests. If the authors have infinite patience it is almost inevitable that they will eventually find a model calibration set up which yields results which they find pleasing. However, this setup is likely to be particular honed to the particular characteristics of the data they have used and is likely to perform less well as other data become available.

Hide

RR by Anonymous Referee #3 (24 Oct 2017)

ED: Publish subject to minor revisions (review by editor) (29 Oct 2017) by Mario Parise

AR by Simon Brenner on behalf of the Authors (22 Nov 2017) Author's response Manuscript

ED: Publish subject to minor revisions (review by editor) (29 Nov 2017) by Mario Parise

AR by Simon Brenner on behalf of the Authors (30 Nov 2017) Author's response Manuscript

ED: Publish as is (15 Dec 2017) by Mario Parise

AR by Simon Brenner on behalf of the Authors (23 Dec 2017)

Short summary

In this study we simulate groundwater levels with a semi-distributed karst model. Using a percentile approach we can assess the number of days exceeding or falling below selected groundwater level percentiles. We show that our approach is able to predict groundwater levels across all considered timescales up to the 75th percentile. We then use our approach to assess future changes in groundwater dynamics and show that projected climate changes may lead to generally lower groundwater levels.