<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "https://jats.nlm.nih.gov/nlm-dtd/publishing/3.0/journalpublishing3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="3.0" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher">NHESS</journal-id>
<journal-title-group>
<journal-title>Natural Hazards and Earth System Sciences</journal-title>
<abbrev-journal-title abbrev-type="publisher">NHESS</abbrev-journal-title>
<abbrev-journal-title abbrev-type="nlm-ta">Nat. Hazards Earth Syst. Sci.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="epub">1684-9981</issn>
<publisher><publisher-name>Copernicus Publications</publisher-name>
<publisher-loc>Göttingen, Germany</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.5194/nhess-14-189-2014</article-id>
<title-group>
<article-title>A quality assessment framework for natural hazard event documentation: application to trans-basin flood reports in Germany</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Uhlemann</surname>
<given-names>S.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff3">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Thieken</surname>
<given-names>A. H.</given-names>
<ext-link>https://orcid.org/0000-0001-7068-2615</ext-link>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Merz</surname>
<given-names>B.</given-names>
<ext-link>https://orcid.org/0000-0002-5992-1440</ext-link>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
</contrib>
</contrib-group><aff id="aff1">
<label>1</label>
<addr-line>Helmholtz Centre Potsdam &amp;ndash; GFZ German Research Centre for Geosciences, Section Hydrology, Potsdam, Germany</addr-line>
</aff>
<aff id="aff2">
<label>2</label>
<addr-line>University of Potsdam, Institute of Earth and Environmental Sciences, Potsdam, Germany</addr-line>
</aff>
<aff id="aff3">
<label>3</label>
<addr-line>now at: ASPEN Re, Aspen Insurance UK Limited, Research and Development, Zurich, Switzerland</addr-line>
</aff>
<pub-date pub-type="epub">
<day>04</day>
<month>02</month>
<year>2014</year>
</pub-date>
<volume>14</volume>
<issue>2</issue>
<fpage>189</fpage>
<lpage>208</lpage>
<permissions>
<copyright-statement>Copyright: &#x000a9; 2014 S. Uhlemann et al.</copyright-statement>
<copyright-year>2014</copyright-year>
<license license-type="open-access">
<license-p>This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit <ext-link ext-link-type="uri"  xlink:href="https://creativecommons.org/licenses/by/3.0/">https://creativecommons.org/licenses/by/3.0/</ext-link></license-p>
</license>
</permissions>
<self-uri xlink:href="https://nhess.copernicus.org/articles/14/189/2014/nhess-14-189-2014.html">This article is available from https://nhess.copernicus.org/articles/14/189/2014/nhess-14-189-2014.html</self-uri>
<self-uri xlink:href="https://nhess.copernicus.org/articles/14/189/2014/nhess-14-189-2014.pdf">The full text article is available as a PDF file from https://nhess.copernicus.org/articles/14/189/2014/nhess-14-189-2014.pdf</self-uri>
<abstract>
<p>Written sources that aim at documenting and analysing a particular natural
hazard event in the recent past are published at vast majority as grey
literature (e.g. as technical reports) and therefore outside of the scholarly
publication routes. In consequence, the application of event-specific
documentation in natural
hazard research has been constrained by barriers in accessibility, concerns
of credibility towards these sources and by limited awareness of their
content and its usefulness for research questions. In this study we address
the concerns of credibility for the first time and present a quality
assessment framework for written sources from a user&apos;s perspective, i.e. we
assess the documents&apos; fitness for use to enhance the understanding of
trans-basin floods in Germany in the period 1952–2002. The framework is
designed to be generally applicable for any natural hazard event
documentation and assesses the quality of a document, addressing
accessibility as well as representational, contextual, and intrinsic
dimensions of quality. We introduce an ordinal scaling scheme to grade the
quality in the individual quality dimensions and the Pedigree score which serves
as a measure for the overall document quality. We present results of an
application of the framework to a set of 133 cases of event-specific
documentation relevant for understanding trans-basin floods in Germany. Our
results show that the majority of flood event-specific reports are of good
quality, i.e. they are well enough drafted, largely accurate and objective,
and contain a substantial amount of information on the sources, pathways and
receptors/consequences of the floods. The validation of our results against
assessments of two independent peers confirms the objectivity and
transparency of the quality assessment framework. Using an example flood
event that occurred in October/November 1998 we demonstrate how the
information from multiple reports can be synthesised.</p>
</abstract>
<counts><page-count count="20"/></counts>
</article-meta>
</front>
<body/>
<back>
<ref-list>
<title>References</title>
<ref id="ref1">
<label>1</label><mixed-citation publication-type="other" xlink:type="simple">Bailin, A. and Grafstein, A.: The critical assessment of research : traditional and new methods of evaluation, Chandos, Oxford, 121 pp., 2010.</mixed-citation>
</ref>
<ref id="ref2">
<label>2</label><mixed-citation publication-type="other" xlink:type="simple">Borenstein, M., Hedges, L. V., Higgins, J. P. T., and Rothstein, H. R.: Introduction to Meta-Analysis, John Wiley &amp; Sons, Ltd., Chichester, UK, 421 pp., 2009</mixed-citation>
</ref>
<ref id="ref3">
<label>3</label><mixed-citation publication-type="other" xlink:type="simple">Bragues, G.: Wiki-philosophizing in a marketplace of ideas: Evaluating Wikipedia&apos;s entries on seven great minds, MediaTropes eJournal, 2, 117–158, 2009.</mixed-citation>
</ref>
<ref id="ref4">
<label>4</label><mixed-citation publication-type="other" xlink:type="simple">Brázdil, R., Kundzewicz, Z. W., and Benito, G.: Historical hydrology for studying flood risk in Europe/L&apos;hydrologie historique pour une meilleure connaissance du risque inondation en Europe, Hydrol. Sci. J., 51, 739–764, 2006.</mixed-citation>
</ref>
<ref id="ref5">
<label>5</label><mixed-citation publication-type="other" xlink:type="simple">Burton, I.: Forensic Disaster Investigations in Depth: A New Case Study Model, Environ. Magazine, 52, 36–41, 2010.</mixed-citation>
</ref>
<ref id="ref6">
<label>6</label><mixed-citation publication-type="other" xlink:type="simple">Centre for Evidence-Based Conservation: Guidelines for Systematic Review in Environmental Management, Environmental Evidence: &lt;a href=&quot;www.environmentalevidence.org&quot;&gt;www.environmentalevidence.org&lt;/a&gt; (last access: 27 September 2012), 2010.</mixed-citation>
</ref>
<ref id="ref7">
<label>7</label><mixed-citation publication-type="other" xlink:type="simple">Cohen, J.: A coefficient of agreement for nominal scales, Educ. Psychol. Meas., 20, 37-46, &lt;a href=&quot;http://dx.doi.org/10.1177/001316446002000104&quot;&gt;https://doi.org/10.1177/001316446002000104&lt;/a&gt;, 1960.</mixed-citation>
</ref>
<ref id="ref8">
<label>8</label><mixed-citation publication-type="other" xlink:type="simple">Farace, D. and Schöpfel, J.: Introduction Grey Literature, in: Grey Literature in Library and Information Studies, edited by: Farace, D. and Schöpfel, J., De Gruyter/Saur, Berlin/New York, 1–7, 2010.</mixed-citation>
</ref>
<ref id="ref9">
<label>9</label><mixed-citation publication-type="other" xlink:type="simple">Fleiss, J. L., and Cohen, J.: The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability, Educ. Psychol. Meas., 33, 613–619, &lt;a href=&quot;http://dx.doi.org/10.1177/001316447303300309&quot;&gt;https://doi.org/10.1177/001316447303300309&lt;/a&gt;, 1973.</mixed-citation>
</ref>
<ref id="ref10">
<label>10</label><mixed-citation publication-type="other" xlink:type="simple">Funtowicz, S. O. and Ravetz, J. R.: Uncertainty and Quality in Science for Policy, Kluwer, Dordrecht, 1990.</mixed-citation>
</ref>
<ref id="ref11">
<label>11</label><mixed-citation publication-type="other" xlink:type="simple">Glaser, R.: Klimageschichte Mitteleuropas, Primus Verlag, Darmstadt, 2001.</mixed-citation>
</ref>
<ref id="ref12">
<label>12</label><mixed-citation publication-type="other" xlink:type="simple">Glaser, R. and Stangl, H.: Historical floods in the Dutch Rhine Delta, Nat. Hazards Earth Syst. Sci., 3, 605–613, &lt;a href=&quot;http://dx.doi.org/10.5194/nhess-3-605-2003&quot;&gt;https://doi.org/10.5194/nhess-3-605-2003&lt;/a&gt;, 2003.</mixed-citation>
</ref>
<ref id="ref13">
<label>13</label><mixed-citation publication-type="other" xlink:type="simple">Gräfe, H.: Ereignisanalyse – Hochwasser August 2002 in den Osterzgebirgsflüssen, Sächsisches Landesamt für Umwelt und Geologie, Dresden, 2004.</mixed-citation>
</ref>
<ref id="ref14">
<label>14</label><mixed-citation publication-type="other" xlink:type="simple">Higgins, J. P. T. and Green, S.: Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 [updated March 2011], The Cochrane Collaboration availabel at: &lt;a href=&quot;http://www.cochrane-handbook.org/&quot;&gt;http://www.cochrane-handbook.org/&lt;/a&gt; (last access: 21 June 2012), 2011.</mixed-citation>
</ref>
<ref id="ref15">
<label>15</label><mixed-citation publication-type="other" xlink:type="simple">Hjørland, B.: Evaluation of an information source illustrated by a case study: Effect of screening for breast cancer, J. Am. Soc. Inf. Sci. Technol., 62, 1892–1898, &lt;a href=&quot;http://dx.doi.org/10.1002/asi.21606&quot;&gt;https://doi.org/10.1002/asi.21606&lt;/a&gt;, 2011.</mixed-citation>
</ref>
<ref id="ref16">
<label>16</label><mixed-citation publication-type="other" xlink:type="simple">Hjørland, B.: Methods for evaluating information sources: An annotated catalogue, J. Informat. Sci., 38, 258–268, &lt;a href=&quot;http://dx.doi.org/10.1177/0165551512439178&quot;&gt;https://doi.org/10.1177/0165551512439178&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref17">
<label>17</label><mixed-citation publication-type="other" xlink:type="simple">Hübl, J., Kienholz, H., and Loipersberger, A. (Eds.): DOMODIS – Documentation of mountain disasters. State of discussion in the european mountain areas, Interpraevent, Schriftenreihe 1, Handbuch 1, Klagenfurt, 40, 2002.</mixed-citation>
</ref>
<ref id="ref18">
<label>18</label><mixed-citation publication-type="other" xlink:type="simple">IRDR: Forensic Investigations of Disasters. The FORIN Project, Integrated Research on Disaster Risk (IRDR), Beijing, available at: &lt;a href=&quot;http://www.irdrinternational.org/2013/01/25/forin-report-1/&quot;&gt;http://www.irdrinternational.org/2013/01/25/forin-report-1/&lt;/a&gt; (last access: 11 October 2012), 2011.</mixed-citation>
</ref>
<ref id="ref19">
<label>19</label><mixed-citation publication-type="other" xlink:type="simple">Kraemer, H. C., Periyakoil, V. S., and Noda, A.: Kappa coefficients in medical research, Statistics in Medicine, 21, 2109–2129, &lt;a href=&quot;http://dx.doi.org/10.1002/sim.1180&quot;&gt;https://doi.org/10.1002/sim.1180&lt;/a&gt;, 2002.</mixed-citation>
</ref>
<ref id="ref20">
<label>20</label><mixed-citation publication-type="other" xlink:type="simple">Lowry, R.: Kappa as a measure of concordance in categorical sorting, available at: &lt;a href=&quot;http://www.vassarstats.net/kappa.html&quot;&gt;http://www.vassarstats.net/kappa.html&lt;/a&gt; (last access: 5 November 2012), 2012.</mixed-citation>
</ref>
<ref id="ref21">
<label>21</label><mixed-citation publication-type="other" xlink:type="simple">MacDonald, B. H., Wells, P. G., Cordes, R., Hutton, G. R. G., Cossarini, D. M., and Soomai, S.: The use and influence of information produced as Grey Literature by international, intergovernmental marine organizations: Overview and current research, in: Grey Literature in Library and Information Studies edited by: Farace, D. and Schöpfel, J., De Gruyter/Saur, Berlin/New York, 167–180, 2010.</mixed-citation>
</ref>
<ref id="ref22">
<label>22</label><mixed-citation publication-type="other" xlink:type="simple">Madnick, S. E., Wang, R. Y., Lee, Y. W., and Zhu, H.: Overview and framework for data and information quality research, ACM J. Data Inf. Quality, 1, 1–22, &lt;a href=&quot;http://dx.doi.org/10.1145/1515693.1516680&quot;&gt;https://doi.org/10.1145/1515693.1516680&lt;/a&gt;, 2009.</mixed-citation>
</ref>
<ref id="ref23">
<label>23</label><mixed-citation publication-type="other" xlink:type="simple">Merz, R. and Blöschl, G.: Flood frequency hydrology: 2. Combining data evidence, Water Resour. Res., 44, W08433, &lt;a href=&quot;http://dx.doi.org/10.1029/2007WR006745&quot;&gt;https://doi.org/10.1029/2007WR006745&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref24">
<label>24</label><mixed-citation publication-type="other" xlink:type="simple">Norris, R., Nichols, S. J., Ransom, G., Webb, A., Stewardson, M., Liston, P., and Mugodo, J.: Causal criteria analysis. Methods manual: a systematic approach to evaluate causality in environmental science, eWater Cooperative Research Centre, Canberra, 2008.</mixed-citation>
</ref>
<ref id="ref25">
<label>25</label><mixed-citation publication-type="other" xlink:type="simple">Osenberg, C. W., Sarnelle, O., and Goldberg, D. E.: Meta-analysis in ecology: Concepts, statistics, and applications, Ecology, 80, 1103–1104, 1999.</mixed-citation>
</ref>
<ref id="ref26">
<label>26</label><mixed-citation publication-type="other" xlink:type="simple">Ranger, S. L.: Grey Literature in Special Libraries: Access and Use in: GL-Conference series, Sixth International Conference on Grey Literature : Work on Grey in Progress, New York Academy of Medicine (USA), 2004.</mixed-citation>
</ref>
<ref id="ref27">
<label>27</label><mixed-citation publication-type="other" xlink:type="simple">Rothstein, H. R. and Hopewell, S.: Grey Literature, in: The handbook of research synthesis and meta-analysis, edited by: Cooper, H. M., Hedges, L. V., and Valentine, J. C., Russell Sage Foundation, New York, 103–125, 2009.</mixed-citation>
</ref>
<ref id="ref28">
<label>28</label><mixed-citation publication-type="other" xlink:type="simple">Samuels, P., and Gouldby, B.: Language of Risk – Project Definitions (Second Edition), FloodSite Project, Document Number: T32-04-01, 2009.</mixed-citation>
</ref>
<ref id="ref29">
<label>29</label><mixed-citation publication-type="other" xlink:type="simple">Seglen, P. O.: Why the impact factor of journals should not be used for evaluating research, Brit, Med. J., 314, 498–502, 1997.</mixed-citation>
</ref>
<ref id="ref30">
<label>30</label><mixed-citation publication-type="other" xlink:type="simple">Simons, K.: The Misused Impact Factor, Science, 322, 165, &lt;a href=&quot;http://dx.doi.org/10.1126/science.1165316&quot;&gt;https://doi.org/10.1126/science.1165316&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref31">
<label>31</label><mixed-citation publication-type="other" xlink:type="simple">Strong, D. M., Lee, Y. W., and Wang, R. Y.: Data quality in context, Commun. ACM, 40, 103–110, &lt;a href=&quot;http://dx.doi.org/10.1145/253769.253804&quot;&gt;https://doi.org/10.1145/253769.253804&lt;/a&gt;, 1997.</mixed-citation>
</ref>
<ref id="ref32">
<label>32</label><mixed-citation publication-type="other" xlink:type="simple">Thieken, A. H., Müller, M., Kreibich, H., and Merz, B.: Flood damage and influencing factors: New insights from the August 2002 flood in Germany, Water Resour. Res., 41, W12430, &lt;a href=&quot;http://dx.doi.org/10.1029/2005wr004177&quot;&gt;https://doi.org/10.1029/2005wr004177&lt;/a&gt;, 2005.</mixed-citation>
</ref>
<ref id="ref33">
<label>33</label><mixed-citation publication-type="other" xlink:type="simple">Thieken, A. H., Seifert, I., Elmer, F., Maiwald, H., Haubrock, S., Schwarz, J., Müller, M., and Seifert, J.: Standardisierte Erfassung und Bewertung von Hochwasserschäden, Hydrol. Wasserbewirts., 53, 198–207, 2009.</mixed-citation>
</ref>
<ref id="ref34">
<label>34</label><mixed-citation publication-type="other" xlink:type="simple">Uhlemann, S.: Supplement to: Data Expansion: The potential of grey literature for understanding floods, Deutsches GeoForschungsZentrum GFZ, &lt;a href=&quot;http://dx.doi.org/10.5880/GFZ.5.4.2012.001&quot;&gt;https://doi.org/10.5880/GFZ.5.4.2012.001&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref35">
<label>35</label><mixed-citation publication-type="other" xlink:type="simple">Uhlemann, S.: Supplement to: A quality assessment framework for natural hazard event documentations: Application to trans-basin flood reports in Germany, Deutsches GeoForschungsZentrum GFZ, &lt;a href=&quot;http://dx.doi.org/10.5880/GFZ.5.4.2013.001&quot;&gt;https://doi.org/10.5880/GFZ.5.4.2013.001&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref36">
<label>36</label><mixed-citation publication-type="other" xlink:type="simple">Uhlemann, S., Thieken, A., and Merz, B.: A consistent set of trans-basin floods in Germany between 1952–2002, Hydrol. Earth Syst. Sci., 14, 1277–1295, &lt;a href=&quot;http://dx.doi.org/10.5194/hess-14-1277-2010&quot;&gt;https://doi.org/10.5194/hess-14-1277-2010&lt;/a&gt;, 2010.</mixed-citation>
</ref>
<ref id="ref37">
<label>37</label><mixed-citation publication-type="other" xlink:type="simple">Uhlemann, S., Bertelmann, R., and Merz, B.: Data expansion: the potential of grey literature for understanding floods, Hydrol. Earth Syst. Sci., 17, 895–911, &lt;a href=&quot;http://dx.doi.org/10.5194/hess-17-895-2013&quot;&gt;https://doi.org/10.5194/hess-17-895-2013&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref38">
<label>38</label><mixed-citation publication-type="other" xlink:type="simple">Van der Sluijs, J. P., Risbey, J. S., and Ravetz, J.: Uncertainty Assessment of Voc Emissions from Paint in the Netherlands Using the Nusap System, Environ. Monitor. Assess., 105, 229–259, 2001.</mixed-citation>
</ref>
<ref id="ref39">
<label>39</label><mixed-citation publication-type="other" xlink:type="simple">Van der Sluijs, J. P., Craye, M., Funtowicz, S., Kloprogge, P., and Ravetz, J.: Combining quantitative and qualitative measures of uncertainty in model-based environmental assessment: The NUSAP system, Risk Anal., 25, 481–492, 2005.</mixed-citation>
</ref>
<ref id="ref40">
<label>40</label><mixed-citation publication-type="other" xlink:type="simple">Wang, R. Y. and Strong, D. M.: Beyond accuracy: what data quality means to data consumers, J. Manag. Inf. Syst., 12, 5–33, 1996.</mixed-citation>
</ref>
<ref id="ref41">
<label>41</label><mixed-citation publication-type="other" xlink:type="simple">Weed, D. L.: Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related, Int. J. Epidem., 29, 387–390, &lt;a href=&quot;http://dx.doi.org/10.1093/ije/29.3.387&quot;&gt;https://doi.org/10.1093/ije/29.3.387&lt;/a&gt;, 2000.</mixed-citation>
</ref>
<ref id="ref42">
<label>42</label><mixed-citation publication-type="other" xlink:type="simple">Weintraub, I.: The role of grey literature in the sciences, available at: &lt;a href=&quot;http://59.67.71.236/download/7D.GreyLit.pdf&quot;&gt;http://59.67.71.236/download/7D.GreyLit.pdf&lt;/a&gt; (last access: 9 November 2010), 2000.</mixed-citation>
</ref>
</ref-list>
</back>
</article>