<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "https://jats.nlm.nih.gov/nlm-dtd/publishing/3.0/journalpublishing3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="3.0" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher">NHESS</journal-id>
<journal-title-group>
<journal-title>Natural Hazards and Earth System Sciences</journal-title>
<abbrev-journal-title abbrev-type="publisher">NHESS</abbrev-journal-title>
<abbrev-journal-title abbrev-type="nlm-ta">Nat. Hazards Earth Syst. Sci.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="epub">1684-9981</issn>
<publisher><publisher-name>Copernicus Publications</publisher-name>
<publisher-loc>Göttingen, Germany</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.5194/nhess-14-2681-2014</article-id>
<title-group>
<article-title>Evaluating data quality collected by volunteers for first-level inspection of hydraulic structures in mountain catchments</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Cortes Arevalo</surname>
<given-names>V. J.</given-names>
<ext-link>https://orcid.org/0000-0003-2551-1942</ext-link>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Charrière</surname>
<given-names>M.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Bossi</surname>
<given-names>G.</given-names>
<ext-link>https://orcid.org/0000-0003-1486-403X</ext-link>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Frigerio</surname>
<given-names>S.</given-names>
<ext-link>https://orcid.org/0000-0002-9664-9773</ext-link>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Schenato</surname>
<given-names>L.</given-names>
<ext-link>https://orcid.org/0000-0003-4573-7383</ext-link>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Bogaard</surname>
<given-names>T.</given-names>
<ext-link>https://orcid.org/0000-0002-1325-024X</ext-link>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Bianchizza</surname>
<given-names>C.</given-names>
</name>
<xref ref-type="aff" rid="aff3">
<sup>3</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Pasuto</surname>
<given-names>A.</given-names>
</name>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Sterlacchini</surname>
<given-names>S.</given-names>
</name>
<xref ref-type="aff" rid="aff4">
<sup>4</sup>
</xref>
</contrib>
</contrib-group><aff id="aff1">
<label>1</label>
<addr-line>Delft University of Technology, Delft, the Netherlands</addr-line>
</aff>
<aff id="aff2">
<label>2</label>
<addr-line>Italian National Research Council, Institute for Geo-hydrological Protection, CNR-IRPI, Padova, Italy</addr-line>
</aff>
<aff id="aff3">
<label>3</label>
<addr-line>Institute of International Sociology Gorizia, ISIG, Gorizia, Italy</addr-line>
</aff>
<aff id="aff4">
<label>4</label>
<addr-line>Italian National Research Council, Institute for the Dynamic of Environmental Processes, CNR-IDPA, Milan, Italy</addr-line>
</aff>
<pub-date pub-type="epub">
<day>06</day>
<month>10</month>
<year>2014</year>
</pub-date>
<volume>14</volume>
<issue>10</issue>
<fpage>2681</fpage>
<lpage>2698</lpage>
<permissions>
<copyright-statement>Copyright: &#x000a9; 2014 V. J. Cortes Arevalo et al.</copyright-statement>
<copyright-year>2014</copyright-year>
<license license-type="open-access">
<license-p>This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit <ext-link ext-link-type="uri"  xlink:href="https://creativecommons.org/licenses/by/3.0/">https://creativecommons.org/licenses/by/3.0/</ext-link></license-p>
</license>
</permissions>
<self-uri xlink:href="https://nhess.copernicus.org/articles/14/2681/2014/nhess-14-2681-2014.html">This article is available from https://nhess.copernicus.org/articles/14/2681/2014/nhess-14-2681-2014.html</self-uri>
<self-uri xlink:href="https://nhess.copernicus.org/articles/14/2681/2014/nhess-14-2681-2014.pdf">The full text article is available as a PDF file from https://nhess.copernicus.org/articles/14/2681/2014/nhess-14-2681-2014.pdf</self-uri>
<abstract>
<p>Volunteers have been trained to perform first-level inspections of hydraulic
structures within campaigns promoted by civil protection of Friuli Venezia
Giulia (Italy). Two inspection forms and a learning session were prepared to
standardize data collection on the functional status of bridges and check
dams. In all, 11 technicians and 25 volunteers inspected a maximum of six structures in
Pontebba, a mountain community within the Fella Basin. Volunteers included civil-protection volunteers, geosciences and social sciences students. Some
participants carried out the inspection without attending the learning
session. Thus, we used the mode of technicians in the learning group to
distinguish accuracy levels between volunteers and technicians. Data quality
was assessed by their accuracy, precision and completeness. We assigned
ordinal scores to the rating scales in order to get an indication of the
structure status. We also considered performance and feedback of participants
to identify corrective actions in survey procedures. Results showed that
volunteers could perform comparably to technicians, but only with a given range in
precision. However, a completeness ratio (question/parameter) was still
needed any time volunteers used unspecified options. Then, volunteers&apos; ratings
could be considered as preliminary assessments without replacing other
procedures. Future research should consider advantages of mobile applications
for data-collection methods.</p>
</abstract>
<counts><page-count count="18"/></counts>
<funding-group>
<award-group id="gs1">
<funding-source></funding-source>
<award-id>263953</award-id>
</award-group>
</funding-group>
</article-meta>
</front>
<body/>
<back>
<ref-list>
<title>References</title>
<ref id="ref1">
<label>1</label><mixed-citation publication-type="other" xlink:type="simple">ADBVE (Autorità di bacino dei fiumi dell&apos;Alto Adriatico): Progetto di Piano Stralcio per l&apos;Assetto Idrogeologico del bacino idrogeografico del fiume Fella, PAI–FELLA, available at: &lt;a href=&quot;http://pai.adbve.it/PAI_Fella/index_fella.html&quot;&gt;http://pai.adbve.it/PAI_Fella/index_fella.html&lt;/a&gt;, last access: 7 July 2014, 2012 (in Italian).</mixed-citation>
</ref>
<ref id="ref2">
<label>2</label><mixed-citation publication-type="other" xlink:type="simple">Bjorkland, R., Pringle, C., and Newton, B.: A Stream Visual Assessment Protocol (SVAP) for Riparian landowners, Environ. Monit. Assess., 68, 99–125, &lt;a href=&quot;http://dx.doi.org/10.1023/A:1010743124570&quot;&gt;https://doi.org/10.1023/A:1010743124570&lt;/a&gt;, 2001.</mixed-citation>
</ref>
<ref id="ref3">
<label>3</label><mixed-citation publication-type="other" xlink:type="simple">Bonney, R., Cooper, C. B., Dickinson, J. L., Kelling, S., Phillips, T., Rosenberg, K. V., and Shirk, J.: Citizen science: a developing tool for expanding science knowledge and scientific literacy, BioScience, 59, 977–984, &lt;a href=&quot;http://dx.doi.org/10.1525/bio.2009.59.11.9&quot;&gt;https://doi.org/10.1525/bio.2009.59.11.9&lt;/a&gt;, 2009.</mixed-citation>
</ref>
<ref id="ref4">
<label>4</label><mixed-citation publication-type="other" xlink:type="simple">Bordogna, G., Carrara, P., Criscuolo, L., Pepe, M., and Rampini, A.: A linguistic decision making approach to assess the quality of volunteer geographic information for citizen science, Information Sciences, 258, 312–327, &lt;a href=&quot;http://dx.doi.org/10.1016/j.ins.2013.07.013&quot;&gt;https://doi.org/10.1016/j.ins.2013.07.013&lt;/a&gt;, 2014.</mixed-citation>
</ref>
<ref id="ref5">
<label>5</label><mixed-citation publication-type="other" xlink:type="simple">Borga, M., Boscolo, P., Zanon, F., and Sangati, M.: Hydrometeorological Analysis of the 29 August 2003 Flash Flood in the Eastern Italian Alps, J. Hydrometeorol., 8, 1049–1067, &lt;a href=&quot;http://dx.doi.org/10.1175/JHM593.1&quot;&gt;https://doi.org/10.1175/JHM593.1&lt;/a&gt;, 2007.</mixed-citation>
</ref>
<ref id="ref6">
<label>6</label><mixed-citation publication-type="other" xlink:type="simple">Brandon, A., Spyreas, G., Molano-Flores, B., Caroll, C., and Ellis, J.: Can volunteers provide reliable data for forest vegetation surveys?, National Areas Journal, 23, 254–262, 2003.</mixed-citation>
</ref>
<ref id="ref7">
<label>7</label><mixed-citation publication-type="other" xlink:type="simple">Burke Engineering, C. B.: Indiana Drainage Handbook, an Administrative and Technical Guide for Activities within Indiana Streams, CBBEL, Indianapolis, Indiana, available at: &lt;a href=&quot;http://www.in.gov/dnr/water/files/allhbook.pdf&quot;&gt;http://www.in.gov/dnr/water/files/allhbook.pdf&lt;/a&gt;, last access: 4 July 2013, 1999.</mixed-citation>
</ref>
<ref id="ref8">
<label>8</label><mixed-citation publication-type="other" xlink:type="simple">Calligaris, C. and Zini, L.: Debris Flow Phenomena: A Short Overview?, in: Earth Sciences, edited by: Imran Ahmad Dar, InTech, ISBN: 978-953-307-861-8, &lt;a href=&quot;http://dx.doi.org/10.5772/29786&quot;&gt;https://doi.org/10.5772/29786&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref9">
<label>9</label><mixed-citation publication-type="other" xlink:type="simple">Cifelli, R., Doesken, N., Kennedy, P., Carey, L. D., Rutledge, S. A., Gimmestad, C., and Depue, T.: The Community Collaborative Rain, Hail, and Snow Network: informal education for scientists and citizens, B. Am. Meteorol. Soc., 86, 1069–1077, &lt;a href=&quot;http://dx.doi.org/10.1175/BAMS-86-8-1069&quot;&gt;https://doi.org/10.1175/BAMS-86-8-1069&lt;/a&gt;, 2005.</mixed-citation>
</ref>
<ref id="ref10">
<label>10</label><mixed-citation publication-type="other" xlink:type="simple">Conrad, C. and Hilchey, K.: A review of citizen science and community-based environmental monitoring: issues and opportunities, Environ. Monit. Assess., 176, 273–291, &lt;a href=&quot;http://dx.doi.org/10.1007/s10661-010-1582-5&quot;&gt;https://doi.org/10.1007/s10661-010-1582-5&lt;/a&gt;, 2011.</mixed-citation>
</ref>
<ref id="ref11">
<label>11</label><mixed-citation publication-type="other" xlink:type="simple">Couvet, D., Jiguet, F., Julliard, R., Levrel, H., and Teyssedre, A.: Enhancing citizen contributions to biodiversity science and public policy, Interdiscipl. Sci. Rev., 33, 95–103, &lt;a href=&quot;http://dx.doi.org/10.1179/030801808X260031&quot;&gt;https://doi.org/10.1179/030801808X260031&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref12">
<label>12</label><mixed-citation publication-type="other" xlink:type="simple">Crall, A. W., Newman, G. J., Jarnevich, C. S., Stohlgren, T. J., Waller, D. M., and Graham, J.: Improving and integrating data on invasive species collected by citizen scientists, Biol. Invasions, 12, 3419–3428, &lt;a href=&quot;http://dx.doi.org/10.1007/s10530-010-9740-9&quot;&gt;https://doi.org/10.1007/s10530-010-9740-9&lt;/a&gt;, 2010.</mixed-citation>
</ref>
<ref id="ref13">
<label>13</label><mixed-citation publication-type="other" xlink:type="simple">Danielsen, F., Burgess, N. D., and Balmford, A.: Monitoring matters: examining the potential of locally-based approaches, Biodivers. Conserv., 14, 2507–2542, &lt;a href=&quot;http://dx.doi.org/10.1007/s10531-005-8375-0&quot;&gt;https://doi.org/10.1007/s10531-005-8375-0&lt;/a&gt;, 2005.</mixed-citation>
</ref>
<ref id="ref14">
<label>14</label><mixed-citation publication-type="other" xlink:type="simple">de Jong, C.: Linking ICT and society in early warning and adaptation to hydrological extremes in mountains, Nat. Hazards Earth Syst. Sci., 13, 2253–2270, &lt;a href=&quot;http://dx.doi.org/10.5194/nhess-13-2253-2013&quot;&gt;https://doi.org/10.5194/nhess-13-2253-2013&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref15">
<label>15</label><mixed-citation publication-type="other" xlink:type="simple">Devictor, V., Whittaker, R. J., and Beltrame, C.: Beyond scarcity: citizen science programmes as useful tools for conservation biogeography, Divers. Distrib., 16, 354–362, &lt;a href=&quot;http://dx.doi.org/10.1111/j.1472-4642.2009.00615.x&quot;&gt;https://doi.org/10.1111/j.1472-4642.2009.00615.x&lt;/a&gt;, 2010.</mixed-citation>
</ref>
<ref id="ref16">
<label>16</label><mixed-citation publication-type="other" xlink:type="simple">Dirksen, J., Clemens, F. H. L. R., Korving, H., Cherqui, F., Le Gauffre, P., Ertl, T., Plihal, H., Müller, K., and Snaterse, C. T. M.: The consistency of visual sewer inspection data, Struct. Infrastruct. E., 9, 214–228, &lt;a href=&quot;http://dx.doi.org/10.1080/15732479.2010.541265&quot;&gt;https://doi.org/10.1080/15732479.2010.541265&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref17">
<label>17</label><mixed-citation publication-type="other" xlink:type="simple">Enders, J.: Measuring community awareness and preparedness for emergencies, Australian Journal of Emergency Management, 16, 52–58, 2001.</mixed-citation>
</ref>
<ref id="ref18">
<label>18</label><mixed-citation publication-type="other" xlink:type="simple">Engel, S. R. and Voshell Jr., J. R.: Volunteer biological monitoring: can it accurately assess the ecological condition of streams?, American Entomologist, 48, 164–177, 2002.</mixed-citation>
</ref>
<ref id="ref19">
<label>19</label><mixed-citation publication-type="other" xlink:type="simple">EPA: Volunteer Stream Monitoring: A Methods Manual, United States Environmental Protection Agency, Washington, DC, available at: &lt;a href=&quot;http://water.epa.gov/type/rsl/monitoring/upload/2002_08_13_volunteer_stream_stream.pdf&quot;&gt;http://water.epa.gov/type/rsl/monitoring/upload/2002_08_13_volunteer_stream_stream.pdf&lt;/a&gt;, last access: 9 February 2014, 1997.</mixed-citation>
</ref>
<ref id="ref20">
<label>20</label><mixed-citation publication-type="other" xlink:type="simple">European Commission: Directive 2007/60/EC of the European Parliament and of the Council of 23 October 2007 on the Assessment and Management of Flood Risks, available at: &lt;a href=&quot;http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32007L0060&quot;&gt;http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32007L0060&lt;/a&gt;, last access: 9 February 2014, 2007.</mixed-citation>
</ref>
<ref id="ref21">
<label>21</label><mixed-citation publication-type="other" xlink:type="simple">European Commission: Civil Protection – Community Co-operation in the Field of Civil Protection, Humanitarian Aid and Civil Protection, available at: &lt;a href=&quot;http://ec.europa.eu/echo/files/civil_protection/vademecum/menu/3.html&quot;&gt;http://ec.europa.eu/echo/files/civil_protection/vademecum/menu/3.html&lt;/a&gt;, last access: 9 February 2014, 2012.</mixed-citation>
</ref>
<ref id="ref22">
<label>22</label><mixed-citation publication-type="other" xlink:type="simple">Flanagin, A. J. and Metzger, M. J.: The credibility of volunteered geographic information, GeoJournal, 72, 137–148, &lt;a href=&quot;http://dx.doi.org/10.1007/s10708-008-9188-y&quot;&gt;https://doi.org/10.1007/s10708-008-9188-y&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref23">
<label>23</label><mixed-citation publication-type="other" xlink:type="simple">Fore, L. S., Paulsen, K., and O&apos;Laughlin, K.: Assessing the performance of volunteers in monitoring streams, Freshwater Biol., 46, 109–123, &lt;a href=&quot;http://dx.doi.org/10.1111/j.1365-2427.2001.00640.x&quot;&gt;https://doi.org/10.1111/j.1365-2427.2001.00640.x&lt;/a&gt;, 2001.</mixed-citation>
</ref>
<ref id="ref24">
<label>24</label><mixed-citation publication-type="other" xlink:type="simple">Galloway, A. W. E., Tudor, M. T., and Haegen, W. M. V.: The reliability of citizen science: a case study of Oregon White Oak stand surveys, Wildlife Soc. B., 34, 1425–1429, &lt;a href=&quot;http://dx.doi.org/10.2193/0091-7648(2006)34[1425:TROCSA]2.0.CO;2&quot;&gt;https://doi.org/10.2193/0091-7648(2006)34[1425:TROCSA]2.0.CO;2&lt;/a&gt;, 2006.</mixed-citation>
</ref>
<ref id="ref25">
<label>25</label><mixed-citation publication-type="other" xlink:type="simple">Gollan, J., Bruyn, L. L., Reid, N., and Wilkie, L.: Can volunteers collect data that are comparable to professional scientists?, a study of variables used in monitoring the outcomes of ecosystem rehabilitation, Environ. Manage., 50, 969–978, &lt;a href=&quot;http://dx.doi.org/10.1007/s00267-012-9924-4&quot;&gt;https://doi.org/10.1007/s00267-012-9924-4&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref26">
<label>26</label><mixed-citation publication-type="other" xlink:type="simple">Gommerman, L. and Monroe, M. C.: Lessons Learned from Evaluations of Citizen Science Programs, School of Forest Resources and Conservation, Florida Cooperative Extension Service, Institute of Food and Agricultural Sciences (IFAS), University of Florida, (FOR291), available at: &lt;a href=&quot;http://edis.ifas.ufl.edu/fr359&quot;&gt;http://edis.ifas.ufl.edu/fr359&lt;/a&gt;, last access: 9 February 2014, 2012.</mixed-citation>
</ref>
<ref id="ref27">
<label>27</label><mixed-citation publication-type="other" xlink:type="simple">Goodchild, M. F.: Citizens as sensors: the world of volunteered geography, GeoJournal, 69, 211–221, &lt;a href=&quot;http://dx.doi.org/10.1007/s10708-007-9111-y&quot;&gt;https://doi.org/10.1007/s10708-007-9111-y&lt;/a&gt;, 2007.</mixed-citation>
</ref>
<ref id="ref28">
<label>28</label><mixed-citation publication-type="other" xlink:type="simple">Goodchild, M. F. and Li, L.: Assuring the quality of volunteered geographic information, Spatial Statistics, 1, 110–120, &lt;a href=&quot;http://dx.doi.org/10.1016/j.spasta.2012.03.002&quot;&gt;https://doi.org/10.1016/j.spasta.2012.03.002&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref29">
<label>29</label><mixed-citation publication-type="other" xlink:type="simple">Gouveia, C. and Fonseca, A.: New approaches to environmental monitoring: the use of ICT to explore volunteered geographic information, GeoJournal, 72, 185–197, &lt;a href=&quot;http://dx.doi.org/10.1007/s10708-008-9183-3&quot;&gt;https://doi.org/10.1007/s10708-008-9183-3&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref30">
<label>30</label><mixed-citation publication-type="other" xlink:type="simple">Gouveia, C., Fonseca, A., Câmara, A., and Ferreira, F.: Promoting the use of environmental data collected by concerned citizens through information and communication technologies, J. Environ. Manage., 71, 135–154, &lt;a href=&quot;http://dx.doi.org/10.1016/j.jenvman.2004.01.009&quot;&gt;https://doi.org/10.1016/j.jenvman.2004.01.009&lt;/a&gt;, 2004.</mixed-citation>
</ref>
<ref id="ref31">
<label>31</label><mixed-citation publication-type="other" xlink:type="simple">Holub, M. and Fuchs, S.: Mitigating mountain hazards in Austria – legislation, risk transfer, and awareness building, Nat. Hazards Earth Syst. Sci., 9, 523–537, &lt;a href=&quot;http://dx.doi.org/10.5194/nhess-9-523-2009&quot;&gt;https://doi.org/10.5194/nhess-9-523-2009&lt;/a&gt;, 2009.</mixed-citation>
</ref>
<ref id="ref32">
<label>32</label><mixed-citation publication-type="other" xlink:type="simple">Holub, M. and Hübl, J.: Local protection against mountain hazards – state of the art and future needs, Nat. Hazards Earth Syst. Sci., 8, 81–99, &lt;a href=&quot;http://dx.doi.org/10.5194/nhess-8-81-2008&quot;&gt;https://doi.org/10.5194/nhess-8-81-2008&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref33">
<label>33</label><mixed-citation publication-type="other" xlink:type="simple">Hudson-Smith, A., Batty, M., Crooks, A., and Milton, R.: Mapping for the masses: accessing Web 2.0 through crowdsourcing, in: Working Paper Series – University College of London for Advanced Spatial Analysis, Centre for Advanced Spatial Analysis University College London, London, UK, available at: &lt;a href=&quot;http://eprints.ucl.ac.uk/15198/1/15198.pdf&quot;&gt;http://eprints.ucl.ac.uk/15198/1/15198.pdf&lt;/a&gt;, last access: 9 February 2014, Paper 143, 1–18, 2008.</mixed-citation>
</ref>
<ref id="ref34">
<label>34</label><mixed-citation publication-type="other" xlink:type="simple">Jakob, M. and Hungr, O.: Debris Flow Phenomena, Praxis, Berlin, 2005.</mixed-citation>
</ref>
<ref id="ref35">
<label>35</label><mixed-citation publication-type="other" xlink:type="simple">Jordan, R. C., Brooks, W. R., Howe, D. V., and Ehrenfeld, J. G.: Evaluating the performance of volunteers in mapping invasive plants in public conservation lands, Environ. Manage., 49, 425–434, &lt;a href=&quot;http://dx.doi.org/10.1007/s00267-011-9789-y&quot;&gt;https://doi.org/10.1007/s00267-011-9789-y&lt;/a&gt;, 2011.</mixed-citation>
</ref>
<ref id="ref36">
<label>36</label><mixed-citation publication-type="other" xlink:type="simple">Mazzorana, B., Hübl, J., Zischg, A., and Largiader, A.: Modelling woody material transport and deposition in alpine rivers, Nat. Hazards, 56, 425–449, &lt;a href=&quot;http://dx.doi.org/10.1007/s11069-009-9492-y&quot;&gt;https://doi.org/10.1007/s11069-009-9492-y&lt;/a&gt;, 2010.</mixed-citation>
</ref>
<ref id="ref37">
<label>37</label><mixed-citation publication-type="other" xlink:type="simple">Molinari, D., Menoni, S., Aronica, G. T., Ballio, F., Berni, N., Pandolfo, C., Stelluti, M., and Minucci, G.: Ex post damage assessment: an Italian experience, Nat. Hazards Earth Syst. Sci., 14, 901–916, &lt;a href=&quot;http://dx.doi.org/10.5194/nhess-14-901-2014&quot;&gt;https://doi.org/10.5194/nhess-14-901-2014&lt;/a&gt;, 2014.</mixed-citation>
</ref>
<ref id="ref38">
<label>38</label><mixed-citation publication-type="other" xlink:type="simple">Najafi, M. and Bhattachar, D. V.: Development of a culvert inventory and inspection framework for asset management of road structures, Journal of King Saud University - Science, 23, 243–254, &lt;a href=&quot;http://dx.doi.org/10.1016/j.jksus.2010.11.001&quot;&gt;https://doi.org/10.1016/j.jksus.2010.11.001&lt;/a&gt;, 2011.</mixed-citation>
</ref>
<ref id="ref39">
<label>39</label><mixed-citation publication-type="other" xlink:type="simple">Newman, G., Zimmerman, D., Crall, A., Laituri, M., Graham, J., and Stapel, L.: User-friendly web mapping: lessons from a citizen science website, Int. J. Geogr. Inf. Sci., 24, 1851–1869, &lt;a href=&quot;http://dx.doi.org/10.1080/13658816.2010.490532&quot;&gt;https://doi.org/10.1080/13658816.2010.490532&lt;/a&gt;, 2010.</mixed-citation>
</ref>
<ref id="ref40">
<label>40</label><mixed-citation publication-type="other" xlink:type="simple">Newman, G., Wiggins, A., Crall, A., Graham, E., Newman, S., and Crowston, K.: The future of citizen science: emerging technologies and shifting paradigms, Front. Ecol. Environ., 10, 298–304, &lt;a href=&quot;http://dx.doi.org/10.1890/110294&quot;&gt;https://doi.org/10.1890/110294&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref41">
<label>41</label><mixed-citation publication-type="other" xlink:type="simple">Nicholson, E., Ryan, J., and Hodgkins, D.: Community data-where does the value lie? Assessing confidence limits of community collected water quality data, Water Sci. Technol., 45, 193–200, 2002.</mixed-citation>
</ref>
<ref id="ref42">
<label>42</label><mixed-citation publication-type="other" xlink:type="simple">Ohio Department of Transportation: Manual of Bridge Inspection, Ohio Department of Transportation, Federal Highway Administration, Ohio, 2010.</mixed-citation>
</ref>
<ref id="ref43">
<label>43</label><mixed-citation publication-type="other" xlink:type="simple">Province of British Columbia: Flood Protection Works Inspection Guide, Water Management Branch, Public Safety Section, British Columbia, Canada, 2000.</mixed-citation>
</ref>
<ref id="ref44">
<label>44</label><mixed-citation publication-type="other" xlink:type="simple">Provinzia Autonoma di Bolzano – Alto Adige: Bestandsaufnahme von Wasserbauwerken der Wildbachverbauung, EF 30 – EF30 Sperrenevaluierung Quick Version, 2006 (in German).</mixed-citation>
</ref>
<ref id="ref45">
<label>45</label><mixed-citation publication-type="other" xlink:type="simple">Protezione Civile della Regione FVG: Formazione, Campus Virtuale, available at: &lt;a href=&quot;http://www.protezionecivile.fvg.it/ProtCiv/default.aspx/81-formazione.htm&quot;&gt;http://www.protezionecivile.fvg.it/ProtCiv/default.aspx/81-formazione.htm&lt;/a&gt;, last access: 7 July 2014, 2009.</mixed-citation>
</ref>
<ref id="ref46">
<label>46</label><mixed-citation publication-type="other" xlink:type="simple">Remaître, A., Malet, J.-P., and Maquaire, O.: Morphology and sedimentology of a complex debris flow in a clay-shale basin, Earth Surf. Proc. Land., 30, 339–348, &lt;a href=&quot;http://dx.doi.org/10.1002/esp.1161&quot;&gt;https://doi.org/10.1002/esp.1161&lt;/a&gt;, 2005.</mixed-citation>
</ref>
<ref id="ref47">
<label>47</label><mixed-citation publication-type="other" xlink:type="simple">Riesch, H. and Potter, C.: Citizen science as seen by scientists: methodological, epistemological and ethical dimensions, Public Underst. Sci., 23, 107–120, &lt;a href=&quot;http://dx.doi.org/10.1177/0963662513497324&quot;&gt;https://doi.org/10.1177/0963662513497324&lt;/a&gt;, 2014.</mixed-citation>
</ref>
<ref id="ref48">
<label>48</label><mixed-citation publication-type="other" xlink:type="simple">Rinderer, M., Kollegger, A., Fischer, B. M. C., Stähli, M., and Seibert, J.: Sensing with boots and trousers – qualitative field observations of shallow soil moisture patterns, Hydrol. Process., 26, 4112–4120, &lt;a href=&quot;http://dx.doi.org/10.1002/hyp.9531&quot;&gt;https://doi.org/10.1002/hyp.9531&lt;/a&gt;, 2012.</mixed-citation>
</ref>
<ref id="ref49">
<label>49</label><mixed-citation publication-type="other" xlink:type="simple">Savan, B., Morgan, A. J., and Gore, C.: Volunteer environmental monitoring and the role of the universities: the case of citizens&apos; environment watch, Environ. Manage., 31, 561–568, &lt;a href=&quot;http://dx.doi.org/10.1007/s00267-002-2897-y&quot;&gt;https://doi.org/10.1007/s00267-002-2897-y&lt;/a&gt;, 2003.</mixed-citation>
</ref>
<ref id="ref50">
<label>50</label><mixed-citation publication-type="other" xlink:type="simple">Seeger, C. J.: The role of facilitated volunteered geographic information in the landscape planning and site design process, GeoJournal, 72, 199–213, &lt;a href=&quot;http://dx.doi.org/10.1007/s10708-008-9184-2&quot;&gt;https://doi.org/10.1007/s10708-008-9184-2&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref51">
<label>51</label><mixed-citation publication-type="other" xlink:type="simple">Servizio Forestale FVG: Catasto Opere Iidraulico Forestali, Manuale Tecnico, Allegato A. Schede di Rilevo, 2002.</mixed-citation>
</ref>
<ref id="ref52">
<label>52</label><mixed-citation publication-type="other" xlink:type="simple">Snäll, T., Kindvall, O., Nilsson, J., and Pärt, T.: Evaluating citizen-based presence data for bird monitoring, Biol. Conserv., 144, 804–810, &lt;a href=&quot;http://dx.doi.org/10.1016/j.biocon.2010.11.010&quot;&gt;https://doi.org/10.1016/j.biocon.2010.11.010&lt;/a&gt;, 2011.</mixed-citation>
</ref>
<ref id="ref53">
<label>53</label><mixed-citation publication-type="other" xlink:type="simple">Tweddle, J., Robinson, L., Pocock, M., and Roy, H.: Guide to Citizen Science: Developing, Implementing and Evaluating Citizen Science to Study Biodiversity and the Environment in the UK, Natural History Museum, London, 2012.</mixed-citation>
</ref>
<ref id="ref54">
<label>54</label><mixed-citation publication-type="other" xlink:type="simple">United Nations: Hyogo Framework for Action 2005–2015: Building the Resilience of Nations and Communities to Disasters, Kobe, Japan, 18–22, available at: &lt;a href=&quot;http://www.unisdr.org/2005/wcdr/intergover/official-doc/L-docs/Hyogo-framework-for-action-english.pdf&quot;&gt;http://www.unisdr.org/2005/wcdr/intergover/official-doc/L-docs/Hyogo-framework-for-action-english.pdf&lt;/a&gt;, last access: 9 February 2014, 2005.</mixed-citation>
</ref>
<ref id="ref55">
<label>55</label><mixed-citation publication-type="other" xlink:type="simple">Uzielli, M., Nadim, F., Lacasse, S., and Kaynia, A. M.: A conceptual framework for quantitative estimation of physical vulnerability to landslides, Eng. Geol., 102, 251–256, &lt;a href=&quot;http://dx.doi.org/10.1016/j.enggeo.2008.03.011&quot;&gt;https://doi.org/10.1016/j.enggeo.2008.03.011&lt;/a&gt;, 2008.</mixed-citation>
</ref>
<ref id="ref56">
<label>56</label><mixed-citation publication-type="other" xlink:type="simple">von Maravic, P.: Evaluation of the Physical Vulnerability of Check Dams Exposed to the Impact of Torrential Processes Through Experimental Analysis, Msc. thesis, University of Padua, Faculty of Agriculture, Departments of Land and Agro-forestry Systems, 2010.</mixed-citation>
</ref>
<ref id="ref57">
<label>57</label><mixed-citation publication-type="other" xlink:type="simple">Yetman, K. T.: Using Maryland&apos;s stream corrido survey to prioritize watershed restoration efforts, J. Am. Water Resour. As., 38, 905–914, &lt;a href=&quot;http://dx.doi.org/10.1111/j.1752-1688.2002.tb05533.x&quot;&gt;https://doi.org/10.1111/j.1752-1688.2002.tb05533.x&lt;/a&gt;, 2002.</mixed-citation>
</ref>
</ref-list>
</back>
</article>