The use of questionnaires for acquiring information on public perception of natural hazards and risk mitigation – a review of current knowledge and practice

Questionnaires are popular and fundamental tools for acquiring information on public knowledge and perception of natural hazards. Questionnaires can provide valuable information to emergency management agencies for developing risk management procedures. Although many natural hazards researchers describe results generated from questionnaires, few explain the techniques used for their development and implementation. Methodological detail should include, as a minimum, response format (open/closed questions), mode of delivery, sampling technique, response rate and access to the questionnaire to allow reproduction of or comparison with similar studies. This article reviews current knowledge and practice for developing and implementing questionnaires. Key features include questionnaire design, delivery mode, sampling techniques and data analysis. In order to illustrate these aspects, a case study examines methods chosen for the development and implementation of questionnaires used to obtain information on knowledge and perception of volcanic hazards in a tourist region in southern Iceland. Face-to-face interviews highlighted certain issues with respect to question structure and sequence. Recommendations are made to overcome these problems before the questionnaires are applied in future research projects. In conclusion, basic steps that should be disclosed in the literature are provided as a checklist to ensure that reliable, replicable and valid results are produced from questionnaire based hazard knowledge and risk perception research. Correspondence to: D. K. Bird (dbird@els.mq.edu.au)


Introduction
The questionnaire is a well established tool within social science research for acquiring information on participant social characteristics, present and past behaviour, standards of behaviour or attitudes and their beliefs and reasons for action with respect to the topic under investigation (Bulmer, 2004).Within natural hazards research, the questionnaire is a popular and fundamental tool for acquiring information on knowledge and perception (Table 1).However, the questionnaire has been relatively neglected in descriptions of social research methods (Bulmer, p. ix, 2004) -and natural hazard studies are no exception.
Research articles should contain sufficient methodological detail to allow reproduction of or comparison with similar studies.This is impossible to achieve if basic criteria are not disclosed in the article.In a review of methodological issues in research articles on risk perception, Hawkes and Rowe (2008) found that most studies using semi-structured questionnaires lacked specific information on question wording and phrasing.Hawkes and Rowe (p. 637, 2008) questioned: "Can we therefore be sure that differences identified in risk perceptions are due to the differences between the people being questioned, or differences in the framing of the questions posed?"In order to overcome this problem, researchers should provide enough detail on important methodological features such as response format (i.e.open or closed questioning), mode of delivery, sampling technique and response rate in peer reviewed research articles.
If the data generated from a questionnaire are to form baseline indicators, then the method has to be comparable over time with identical wording of questions (Enders, 2001).As Cecić and Musson (p. 41, 2004) highlighted "The point of Published by Copernicus Publications on behalf of the European Geosciences Union.having a questionnaire is primarily to have all the data in more or less the same format, which means that all the questions are asked of the whole population of observers in precisely the same way.It makes the collected data comparable within the data set. . . . . .as well as between different events (for which the same type of form was used)".Furthermore, international scientific journals request that authors provide enough methodological detail to allow the work to be reproduced (see Elsevier, 2009).Therefore, a copy of the questionnaire should be provided within the text, as an appendix or available electronically (i.e.referenced in the article as available online or via email from the author).
An extensive review of 46 articles which describe results generated from questionnaires related specifically to natural hazards revealed that very few explained the basic techniques employed for their development and implementation (Table 2).The majority of these articles were sourced from keyword (i.e.questionnaire/survey and natural hazards) searches in relevant scientific literature databases (e.g.Elsevier, Science Direct) in addition to journal specific databases (e.g.Disasters, Natural Hazards and Earth System Sciences, Natural Hazards).References in these articles then provided access to additional sources.Articles that had not undergone peer review were excluded (Drabek, 1986).This included articles that referenced an empirical study (e.g. a working paper or project report) for a more comprehensive description of the methods applied.Other articles were excluded due to ambiguity within their description of methods.
Each article was assessed on the basis of whether or not it included methodological detail on response format, mode of delivery, sampling technique and response rate.These techniques were selected because they are described in many social and behavioural sciences texts (see references in Sect. 3 of this article) as some of the basic methods employed during the development and implementation of a simple questionnaire.Access to the questionnaire (whether it is provided within the article or available electronically) was also noted.
The oldest of these articles was published in 1996 and more than half were published within the last two years.All articles were sourced from international, peer reviewed scientific journals.From these articles 65% reported response format, 57% reported mode of delivery, 37% reported sampling technique, 50% reported response rate and 33% provided access to the questionnaire.Overall, only 9% provided all five of these criteria.One article in a leading natural hazards journal simply stated "a questionnaire regarding. . .was sent to . . ." before presenting the results.If the work has been published elsewhere it is common practice to simply Nat.Hazards Earth Syst.Sci., 9, 1307Sci., 9, -1325Sci., 9, , 2009 www.nat-hazards-earth-syst-sci.net/9/1307/2009/ , response rate, the questionnaire and the total number of articles that include all 5 of these criteria (Akason et al., 2006;Anderson-Berry, 2003;Badri et al., 2006;Barberi et al., 2008;Bird andDominey-Howes, 2006, 2008;Bruen et al., 2008;Carlino et al., 2008;Davis et al., 2005;Dolce and Ricciardi, 2007;Dominey-Howes and Minos-Minopoulos, 2004;Gaillard, 2008;Gaillard et al., 2008;Glatron and Beck, 2008;Gregg et al., 2004aGregg et al., , b, 2006Gregg et al., , 2007;;Grímsdóttir and McClung, 2006;Haynes et al., 2007Haynes et al., , 2008a, b;, b;Johnston and Benton, 1998;King et al., 2006;Kozak et al., 2007;Kreibich et al., 2005;Kurita et al., 2006;Lam et al., 2007;Leonard et al., 2008;Lindell and Whitney, 2000;Martin et al., 2007;McIvor and Paton, 2007;Meheux and Parker, 2006;Paton et al., 2001aPaton et al., , b, 2008a;;Perry and Lindell, 2008;Raaijmakers et al., 2008;Rasid et al., 1996;Solana and Kilburn, 2003;Solana et al., 2008;Thieken et al., 2005Thieken et al., , 2007;;Tran et al., 2008;Xie et al., 2007;Zhai and Ikeda, 2008 reference the original source by stating "a more detailed description of the whole questionnaire can be found in . . .".But frustratingly some of the original articles did not provide this "detailed description".Instead, they refer the reader back to the other article thus creating a circular form of referencing lacking in valuable detail. With growing concern about climate change and its potential effect on increasing natural hazard frequency and magnitude (IPCC, p. 110-111, 2007) the time has come to provide a clear template for questionnaire development and implementation for researchers investigating public knowledge and perception of, and response to, natural hazards.This is because the development of appropriate mitigation and adaptation strategies will not evolve from the physical sciences alone but rather in combination with an understanding of public knowledge and perception of hazard and risk (Anderson-Berry, 2003;Barberi et al., 2008;Bird andDominey-Howes, 2006, 2008;Brilly and Polic, 2005;Dominey-Howes and Minos-Minopoulos, 2004;Gregg et al., 2007;Johnston et al., 2005;Lindell and Whitney, 2000;Solana and Kilburn, 2003) and behaviour when faced with hazards (Chester et al., 2008;Gaillard, 2008;Gaillard et al., 2008;Gregg et al., 2004b;Haynes et al., 2008a;Lavigne et al., 2008;McIvor and Paton, 2007;Paton et al., 2008b).This article reviews current knowledge and practice for developing and implementing questionnaires.Following a brief discussion on approaches to social science research, consideration is given to the key features in developing questionnaires, choice of the most appropriate mode of delivery, employment of sampling techniques, data analysis and piloting the questionnaire.To illustrate these aspects, a case study examines methods chosen for the development and implementation of questionnaires for obtaining information on knowledge and perception of volcanic hazards in a tourist region in southern Iceland.Key findings from this pilot investigation are presented, followed by a review of the questionnaire's design and interview process, and concludes with recommendations for future studies.Finally, a review of survey research on the human dimension of risk and related methodological issues is presented.

Approaches to social science research
Approaches to social research can be qualitative or quantitative.Philosophical assumptions, strategies of enquiry and specific research methods define the variations between the two (Creswell, 2003).Hanson (2008), however, argues that these sociological approaches have converged.Certainly, one can be integrated within the other (e.g.Haynes et al., 2007) in order to strengthen research design (Patton, 1990).This mixed methods approach (Creswell, 2003) may include sequential procedures (  Regardless of whether researchers adopt a qualitative or quantitative approach, some aspects of each will be incorporated into research design (Sarantakos, 2005).With respect to questionnaires, qualitative comments (e.g.generated from open-ended questions) can be used to corroborate, illustrate or elaborate on the meaning of quantitative responses (Bazeley, 2006).The following section provides an overview of key features for the development and implementation of a questionnaire with reference to both qualitative and quantitative approaches.
3 Key features for developing and implementing a questionnaire

Developing a questionnaire
Good questionnaire design is crucial (Bulmer, 2004;Creswell, 2003;de Vaus, 2002;McGuirk and O'Neill, 2005;Oppenheim, 1992;Parfitt, 2005;Patton, 1990;Sarantakos, 2005) in order to generate data conducive to the goals of the research.Questionnaire format, sequence and wording, the inclusion of classification, behavioural, knowledge and perception questions, and questionnaire length and output, need to be considered to ensure reliability, validity and sustained engagement of the participant.
The principal requirement of questionnaire format is that questions are sequenced in a logical order, allowing a smooth transition from one topic to the next (Sarantakos, 2005).This will ensure that participants understand the purpose of the research and they will carefully answer questions to the end of the survey (McGuirk and O'Neill, 2005).This can be accomplished by grouping related questions under a short heading describing the section's theme.
Researchers must decide on question response format.That is, whether to include closed questions, open questions or both.There is debate on the use of open and closed questions within social research (Bulmer, 2004;Vol. 1, Sect. 2).Closed questions are typically difficult to construct but easy to analyse whereas open questions are easy to construct but difficult to analyse (Sarantakos, 2005).Closed questions are often used within quantitative research while open questions are used within qualitative research (Table 3).Closed questions are easy to administer, easily coded and analysed, allow comparisons and quantification, and they are more likely to produce fully completed questionnaires while avoiding irrelevant responses (Sarantakos, 2005).Nominal, ordinal, interval and ratio levels are used to measure degrees of difference in closed questions (Table 4).However, researchers must have a comprehensive understanding of the possible range of participant responses which makes the design of closed questions demanding (de Vaus, 2002).To minimise the effect of limiting participants to predefined answers the options "other, please specify" (McGuirk and O'Neill, 2005), "don't know" or "not applicable" should be included where appropriate (Oppenheim, 1992).Clear instructions should be given which describe how participants are expected to answer closed questions e.g.please choose only one response from the list provided.
Advantages for open-ended questioning include freedom and spontaneity of answers, opportunity to probe and usefulness for testing hypotheses about ideas or awareness (Oppenheim, 1992).Open questions allow time and space for free-form responses which invite participants to share their understandings, experiences, opinions and interpretations of, as well as their reactions to, social processes and situations (McGuirk and O'Neill, 2005).However, given that a large variety of answers may be provided for any one question, analysis of the results can be challenging.With this in mind, open questions can be asked in a style that directs participants into definite channels without actually suggesting responses (Payne, 1951) e.g.how many people are there in your family living at this address?
Overall, a combination of closed and open questions provides the survey write-up with quantifiable and in-depth results.Closed questions produce results that are easily summarised and clearly presented in quick-look summaries while open questions produce verbatim comments adding depth and meaning.
Next to consider is the type of questions to include.Five basic types of questions are classification, behavioural, knowledge, perception and feelings (Table 5).Classification questions related to age, education, occupation and place of residence help place participants in relation to others (Patton, 1990) as well as providing information that may predict the main effects (Parfitt, 2005) revealed from behavioural, knowledge, perception or feeling questions.
To produce reliable and valid results, the wording of each question should be precise and unambiguous to ensure that each participant can interpret its meaning easily and accurately (Payne, 1951).Reliability refers to the consistency of a question; that is, the probability of obtaining the same results if the question is duplicated.Validity refers to whether or not the question measures what it was intended to (Oppenheim, 1992).To achieve reliability and validity, questions should be short, simple and in line with the targeted population's vernacular and avoid problems such as doublebarrelled questions (Table 6).
Once the questionnaire has been designed and formatted researchers should reconsider length.The key rule is that the questionnaire should contain as many questions as necessary and as few as possible (Sarantakos, 2005).Every question should have a clear role and purpose (McGuirk and O'Neill, Table 5.Five basic types of questions that can be asked of a participant (after Patton, p. 290-293, 1990).

Question type Description Example
Classification 2005).Questionnaires should take no longer to complete than participants are willing to spend time answering and the interviewer is able to commit (including the time commitment of data entry and analysis).

Choosing the most appropriate mode of delivery
Each questionnaire mode of delivery has advantages and disadvantages (Table 7) and selection will depend on each mode's suitability to the study and available resources (Oppenheim, 1992).Factors that should be taken into account when selecting delivery mode include sample size and distribution, types of questions, nature of the population, survey topic, availability of resources (e.g.skilled interviewers, equipment, funding) and time constraints (de Vaus, 2002).Initial contact in the form of a cover letter, telephone call or direct approach is the first step to building rapport and motivating participation for any mode of distribution.During first contact, researchers should introduce themselves and their credentials, explain the study and why it is being conducted, reveal why the person was selected for the study, indicate how long the questionnaire will take to complete and the intended use of the results (Dunn, 2005).Due to ethical considerations, participants should be assured that no harm will come to them as a result of their participation and they have the right to anonymity, the right to refuse to answer certain questions and the right to refuse to be interviewed (Oppenheim, 1992).during a certain period are considered for the investigation • Purposive Participants who are thought to be relevant to the research are purposively chosen • Quota A "quota" of participants to be chosen from a specific population group is predetermined

• Snowball
The first participant recommends other people who meet the research criteria Self-administered questionnaires may contain a further introductory paragraph to help set the scene and guide participants towards answering instructions and definitions (Oppenheim, 1992).Not only do these directions clarify questions and procedures but they also serve to maintain motivation.Questionnaire format and graphic layout is especially significant with self-administered modes as it helps promote response rates (Sarantakos, 2005).Eliciting reliable and valid data relies on developing an attractive and professional design.Self-administered questionnaires may also be delivered to participants by someone in an official position.The questionnaire is then left for the participant to complete at their own leisure.
Questionnaires developed for telephone or face-to-face delivery should contain instructions for participants as well as researchers administering the survey (Oppenheim, 1992).Interviewers should be trained to conduct and deliver the questionnaire to ensure that differences between participant responses is a reflection of their knowledge or perception and not on how the data were collected (Collins, 2003).The interviewer should not be in a position to make judgements, include subjective views or personal bias and convictions (Sarantakos, 2005).
To ensure a high rate of participation with an administered questionnaire researchers should consider approach, explanation, respect, trust and friendliness.Sarantakos (p. 288, 2005) provides a list of issues that may be employed to improve interview response rates.More insightful and valid data will generally be gained from telephone and face-toface interviews when both the interviewer and participant feel at ease and when the interviewer adopts an active role rather than just asking a list of predetermined questions (Dunn, 2005).Administered questionnaires may also be delivered to groups of participants assembled together (e.g.focus groups).

Employing sampling techniques
The sampling technique will determine how representative the sample is of the population of interest.In addition to reflecting the population's characteristics such as age, socioeconomic status, education, gender and marital status, a representative sample is one where every member of a population has a statistically equal chance of being selected (Oppenheim, 1992).Probability sampling ( However, both forms of sampling can be either quantitative or qualitative and the use of both is common in mixed methods research (Kemper et al., 2003).
The main factors to consider when determining sample size are: -degree of accuracy (in relation to sampling error and confidence interval) and -the extent of variation in the population in regard to the key characteristic of the study (de Vaus, 2002).
Sampling error reflects the extent to which the sample differs from the population while confidence level indicates the likelihood of the population parameter lying within the specified range.Statistical tables are used to determine sampling error associated with various sample sizes (e.g. de Vaus, p. 81, 2002).However, these assume that an up-to-date and accurate sampling frame (a list containing all members of the target population) was used, the sampling was faultless, biases have been compensated for, field work was precise and that there is no non-response error (Oppenheim, 1992).In summary, the size of a sample will depend upon theoretical requirements (e.g.sampling error, cluster size, required accuracy of population estimates), precision of the sampling operation, number of sub-group comparisons, nature of the dependent variable and temporal and financial constraints.Statistical assumptions about sampling errors do not apply to non-probability sampling (Oppenheim, 1992).

Analysing data
Before data analysis, researchers should address nonresponse error and missing data.Low response rates can be overcome by sending out more questionnaires, careful training of interviewers, use of interpreters and conducting call backs at different times of the day and week.However, these do not overcome the problem of non-response bias.Often non-respondents are characteristically different from responders.To reduce the effect of non-response bias on the analysis, statistical techniques such as statistical weighting and multivariate analysis can be applied (see Oppenheim, p. 106-107, 1992).Alternatively, de Vaus (p. 84, 2002) suggests obtaining information about the non-respondents, using available observational data, gathering information from the sampling frame or exploring known characteristics of the population, in order to neutralise the effect of non-response bias during the analysis.Missing data may be enclosed within an otherwise complete individual record.Techniques such as listwise (deletion of all data for an entire case that has missing entries) and pairwise deletion (partial deletion of a case for only the missing entries) can be employed to deal with missing data (Oppenheim, 1992).However, before undertaking such methods, researchers should consider missing data bias (i.e. if participant characteristics are different between complete datasets and incomplete ones (de Vaus, 2002)).
Raw data from open and closed questions should be converted into numbers for data entry.Questionnaires may be pre-coded, field coded (coded on the spot by the interviewer) or office coded after the questionnaire has been completed.Coding complex open questions for quantitative analysis involves developing categories and assigning corresponding labels and values to each question (Sarantakos, 2005).A code book with coding instructions should be compiled regardless of whether data entry rests with one or several persons.General principles to incorporate within the instructions include how to deal with queries, a description of the case numbering system and the method for coding multiple-response questions, to ensure that the data are entered accurately by all researchers (Oppenheim, 1992).Continuous checks and crosschecks are a critical part of data preparation and quality assurance protocols, in order to certify that "interesting" findings are not simply due to mistakes in data entry.Common practices such as running frequency distributions on the main variables, range checks for each variable and internal consistency checks help identify inconsistencies within datasets (Oppenheim, 1992).
Once the data have been checked, edited, coded and entered, analysis can begin.Data analysis can be undertaken manually or electronically.The latter, known as computer assisted data analysis (CADA) is most commonly used.The most popular program, offering a powerful, fast and reliable statistical analysis for quantitative data, is SPSS ® (Statistical Package for the Social Sciences) (Sarantakos, 2007).SPSS ® can provide a general and descriptive overview of data through "frequency tables" and "crosstabs" while correlation tests assess associations between variables and tests of significance assess whether or not results from the sample can be applied to the population.
Coding verbatim for qualitative analysis may entail one of many methods depending on the approach (e.g.Sarantakos, p. 349-350, 2005).The purpose of coding in qualitative research is to classify and tag text with codes in order to facilitate later retrieval.It is a way of linking data to ideas and from ideas back to supporting data (Bazeley, 2007).Word processing programs such as Microsoft Word ® are adequate for low-level CADA of qualitative data.However, for more complex qualitative analysis, a range of other software allows researchers to process data similarly to that employed Highlighted on the map are the western jökulhlaup hazard zone, an outline of the Katla caldera and the three regional evacuation centres in the towns of Hella, Hvolsvöllur and Skógar.One four-wheel-drive road, crossing 10 glacial rivers and tributaries, gives tourists' access to Þórsmórk.Rather than crossing these fast flowing channels, tourists will be instructed to stay in Þórsmórk during a volcanic crisis.They will not be advised to evacuate to the designated centres.
in quantitative analyses (Sarantakos, 2005).For example, the program NVivo ® allows researchers to undertake projects ranging from fine, deeply reflective analysis to analytical processing of large volumes of data (Bazeley, 2007).Within NVivo ® researchers can manage data and ideas, perform simple or complex queries and model ideas or concepts graphically.
Analysing datasets from a mixed methods approach to identify apparent themes is referred to as triangulation.This encompasses integrating qualitative and quantitative datasets, comparing multiple qualitative datasets and, multiple perspectives from multiple observers.Triangulation helps overcome the intrinsic bias that is inherent within single-method, single-observer and single-theory studies and as such, offers greater validity (Denzin, 2006).Bryman (2006) advocated the combination of quantitative and qualitative research with the argument that despite the apparent problems and concerns of triangulation the advantages are overwhelming.However, Blaikie (2006) concludes that the use of methods drawn from different methodological perspectives is not legitimate within the same study unless they are used sequentially.Similarly, Hemming (2008) prefers to combine qualitative datasets through "crystallisation" in order to build a more holistic picture of the issue under investigation rather than simply duplicating results to improve validity.

Piloting the questionnaire
Before embarking on the main study, the questionnaire should be piloted to test its usefulness in providing valuable information that might be relevant to emergency management personnel for the development of risk mitigation and adaptation strategies (Bird and Dominey-Howes, 2008).The pilot phase is also practical for detecting major defects in questionnaire design.Pilot work can be costly but it will avoid a great deal of wasted effort on unintelligible questions producing unquantifiable responses and uninterpretable results (Oppenheim, 1992).
The next section describes a case study whereby questionnaires were developed and piloted in southern Iceland -with specific reference to volcanic hazards.

Rationale
A future eruption of Katla volcano could cause a jökulhlaup (glacial outburst flood) from the western region of the Mýrdalsjökull ice cap and down the river Markarfljót in southern Iceland (Fig. 1).If this were to occur the tourist destination of Þórsmörk would be the first affected.A catastrophic jökulhlaup (with a discharge>100 000 m 3 s −1 ) on the Markarfljót, triggered by a Katla eruption, would produce a flood height across the floodplain in excess of 20 m, reaching Þórsmörk no more than two hours after the start of the eruption (Guðmundsson et al., 2005).Þórsmörk consists of several valleys and ridges, small mountain hut communities and several hundred kilometres of walking tracks.More than 14 000 overnight stays were recorded during 2006 summer season (Statistics Iceland, 2007, personal communication).
While there is abundant literature on the physical attributes of Icelandic jökulhlaup (e.g.Björnsson et al., 2000;Carrivick, 2007;Eliasson et al., 2006;Guðmundsson et al., 2005;Larsen, 2000Larsen, , 2002;;Roberts, 2005;Rushmer, 2007;Russell et al., 2006;Smith, 2004;Smith and Haraldsson, 2005) little research exists on the social aspects of jökulhlaup hazards and none exists for the tourist region of Þórsmörk.To bridge this gap in understanding, questionnaires were developed and piloted with tourists and tourism employees in Þórsmörk.The aims of this pilot investigation are to: (1) report on the methods chosen for the development and implementation of the questionnaires, (2) briefly summarise the key findings, and (3) review the questionnaire design and interview process making recommendations to improve these in future studies.

Methods chosen for the development and implementation of the questionnaires
Survey design and format was based on a questionnaire developed and tested by Bird and Dominey-Howes (2008).However, some questions were added while others were adjusted or removed from the original questionnaire in order to suit the regional setting and hazards.The purpose of developing and implementing the questionnaire was to generate data that may be useful to emergency management agencies (particularly the Icelandic Civil Protection (ICP)) for developing risk mitigation strategies around Katla.To identify insights and differences in perceptions between stakeholder groups, distinct questionnaires were drafted for tourists and tourism employees.The proposed contents were discussed with key emergency management personnel from the ICP, Iceland Search and Rescue (ICE-SAR) and local police prior to this pilot investigation and minor adjustments were made according to their comments and views.
To produce quantifiable and in-depth results that will be meaningful to emergency management agencies, open and closed questions were incorporated in the design.Check-box answers were provided for certain closed questions with the option "other, please specify" so as not to limit participant responses to pre-defined answers.To gain an in-depth understanding of knowledge and motivations participants were asked "why", or "if yes/no, can you tell me/can you describe. . ." following certain closed questions.Where applicable, open questions were used to avoid leading participants into pre-defined answers and to gather more detailed responses.A large variety of nationalities were expected in this region, so where possible diction was kept simple without the use of academic jargon or complicated expressions.
Demographic data such as participant age and level of education were gathered in the first section of each questionnaire.Country of residency was included since the survey was aimed at both local and international tourists and tourism employees.A series of questions were integrated for both groups to assess participant's self protective behaviour, their knowledge and awareness of Katla and jökulhlaup hazards, perception of jökulhlaup hazards in the Markarfljót region and knowledge and perception of evacuation procedures.To be counted as correct, responses had to include the approximate recurrence interval of Katla eruptions and the year of the last eruption.Their definition of jökulhlaup was counted as correct if participants mentioned something about flood water from a glacier.Additional questions were incorporated for the tourist group to gather information on their length of stay and purpose for visiting Þórsmörk.Extra questions in the tourism employee's group collected data on company training, group characteristics and guiding techniques.Electronic copies of both questionnaires are available from the author.
Due to the length of the questionnaire and the nature of the open/closed questioning face-to-face delivery was considered most appropriate.Administering the questionnaires face-to-face prevented participants from taking time to research "correct" answers (since information on how participants would respond in a sudden emergency situation should be generated from this type of survey) and it allowed the interviewer to be more actively involved in data collection.
All participants were selected through a non-probability purposive sampling technique where potential participants working or staying in the Þórsmörk region were approached directly.These people were selected as it was expected that they had an interest in the region or hazard.Within the context of this investigation people located within the remote region of Þórsmörk who could possibly be one of first groups affected by a Katla eruption were targeted.
Due to the lack of demographic data on Þórsmörk tourists it is impossible to determine whether or not this small sample size is representative of the broader population.All hut wardens located in Þórsmörk were approached during this investigation and a response rate of 89% was achieved.An 80% response rate was recorded for the tourist group.Both these figures are high and may be due to the nature of this pilot investigation and the purposive sampling technique.
To determine a questionnaire's usefulness and suitability a pilot investigation should be conducted with approximately 20 participants (Parfitt, 2005).Twenty-four participants in the tourist group and 16 participants in the tourism employees group were recruited for this investigation.Before conducting the questionnaire, participants were required to read a letter which explained the questionnaire's purpose and content, that they could withdraw from the survey at any time without consequence and that no participant would be identifiable through publication of the results.Their approval of these conditions and consent to conduct the interview was indicated by their signature on this letter.
Each question was read aloud by the interviewer and participants were instructed to respond verbally.The interviewer recorded all answers on the questionnaire with any other relevant information communicated by the participant.All interviews were conducted in English during August and September 2006.This time period was chosen as it falls within the tourist high season and therefore, a broad crosssection of nationalities could be sampled.

Key findings from the questionnaires
Data entry and analysis was carried out using SPSS ® 15.0.Due to the nature of this pilot study only a brief summary of key findings will be given.It is unclear whether or not these results represent the broader tourist sector.However, they provide baseline data against which future research can be assessed.Questions listed in each result table are written exactly as in their corresponding questionnaire.Results from tourist questionnaires are in Table 9.This includes participant responses to two classification questions in addition to various behavioural, knowledge and perception questions.Results from tourism employee questionnaires are in Table 10.This includes participant responses to two classification questions in addition to company information and various behavioural, knowledge and perception questions.

Review of questionnaire design and the interview process and recommendations for improvement
Administering the questionnaires via face-to-face delivery, researchers could determine whether or not participants were comfortable with the sequence and structure of questions, questionnaire length and determine if there were any other defects with its design.In general, questions were sequenced in a logical order.However, in the tourist group questionnaire, questions relating to media discussions on Katla should be placed after the question "have you heard of Katla".If the participant answers "yes", then it is appropriate to ask them if they have followed media discussions about Katla.If they state "no", then these questions can be skipped.
A few problems arose with respect to question wording.The first question was inadequately worded although this was overcome by the interviewer.The question simply read "Age?" instead of writing the full question "Within which age group were you on your last birthday?"If the questionnaire was self-administered, then participants would have to add words in order to make an answerable question and this is not desirable since the object is to have all participants answering the same questions (Fowler, 2002).Problems arose with the structuring of one particular question.The question "Do you know whether a jökulhlaup warning system exists for the Markarfljót region?" was confusing for participants.This question should be rewritten as "Does a jökulhlaup warning system exist for the Markarfljót region?" so participants have the opportunity to state "yes" one does exist or "no" one does not exist or simply "do not know".Overall, the questionnaire took approximately 30 min to complete which was acceptable to the participants.However, some participants took the opportunity to spend more or less time as required.No major defects in questionnaire design were apparent during this pilot investigation.Some natural hazards researchers preferentially use selfadministered questionnaires to prevent participants feeling uncomfortable in front of the interviewer, or natural pressure in giving a "correct" response.However, it is possible that participants may not fully comprehend certain questions (Solana and Kilburn, 2003).If an interviewer is present they may offer assistance if they perceive, through body language or an irrelevant response, that participants do not understand a question.Furthermore, the interviewer will be able to determine whether or not participants comprehend all questions the same way (Collins, 2003).This is a critical factor during the pilot phase as researchers can assess if certain aspects of the questionnaire need to be changed before the main survey is conducted.
Considering that participants within the tourist group came from such varied cultural, ethnical, educational and generational backgrounds it is recommended that future surveys use face-to-face delivery.If the questionnaire is administered only in English then future research should consider non-response bias of non-English speakers.Furthermore, the interviewer needs to ensure that all participants whose first language is not English understand each question as it is intended.It is suggested that future studies consider providing the questionnaire in various languages in order to ensure reliability and validity.However, if a multilingual questionnaire is administered then recruitment of qualified interviewers is essential with appropriate training to reduce bias and error-producing factors, and to encourage accuracy, clarity and inter-interviewer consistency (Sarantakos, 2005).
The purposive sampling technique was considered appropriate for this type of research.However, when considering the temporal distribution of Icelandic tourists in Þórsmörk it would be more representative to conduct interviews throughout the whole year as many visit Þórsmörk during winter.Due to the vast and rugged landscape, targeting potential participants when they were located within the mountain huts was deemed the most logical method for recruitment.If future studies adopt this technique it is advisable to increase the sample size to reduce the effect of over-relying on accessible participants and thereby ensuring a reasonably representative sample (Sarantakos, 2005).
Questionnaires were office coded and data entry and analysis was carried out using SPSS ® and Microsoft Word ® .Categories were developed for various open answer questions to enable analysis in SPSS ® .Due to the small sample size the analysis of the verbatim record was sufficient in Microsoft Word ® .However, if a larger sample size is obtained and qualitative analysis is required then the use of a program such as NVivo ® is suggested.
A simple questionnaire, involving a small sample size purposively selected from the population was used in this pilot investigation.This simple questionnaire was used for two main reasons: (1) to provide a working example of the basic techniques employed for developing and implementing a questionnaire and (2) to create a benchmark for future studies on knowledge and perception of Katla in Þórsmörk since this research is the first of its kind to assess volcanic risk and tourism in southern Iceland.Future studies, which incorporate the suggested improvements to the questionnaire, should encapsulate a more rigorous quantitative design with a larger sample size to produce statistically reliable results that are representative of the tourist sector.
While a simple questionnaire was used in this pilot investigation due to the abovementioned reasons, more comprehensive instruments have been designed based on various theoretical frameworks developed in relation to risk perception and disaster research.The following section provides a brief review of the prominent research in the realm of risk perception and discusses related methodological issues with emphasis on the use of the questionnaire.

Research on the human dimension of risk and methodological issues
G. F. White's (1945) ground-breaking work on human adjustments to floods pioneered the way for research on the human dimension of risk in multi-hazard environments.Nearly three decades later, Kates (1971) described human behaviour as a key component of the present state of natural hazards research within the international community.Kates (1971) illustrated that the choices made by individuals who occupy hazardous areas are related to their perception, awareness and evaluation of the hazard.Douglas and Wildavsky's (1982) view that risk perception and risk-related behaviour is primarily a socio-cultural phenomenon has influenced the way in which questions are asked of participants in survey research.Other researchers (e.g.Slovic, 2000b) have employed the psychometric paradigm.This approach to risk perception research applies "psychophysical scaling and multivariate analysis techniques to produce quantitative representations or "cognitive maps" of risk attitudes and perceptions" (Slovic, p. 222, 2000a).However, the disparity between the public's overreaction to officially designated minimal risks and their under-reaction to adopt preparedness measures despite government warnings, led to the development of the social amplification of risk framework (SARF) (Horlick-Jones et al., 2003).
Introduced in 1988, SARF was developed in an attempt to overcome the fragmented nature of risk perception and risk communication research (Kasperson et al., 2003).To achieve this, an integrative theoretical framework was established to deal with results produced from media research, from the psychometric and cultural schools of risk perception research and, from studies of organisational response to risk.In essence, within the SARF framework, risk experience not only relates to the physical experience of harm but also to the result of processes by which groups and individuals learn to acquire or create interpretations of risk, whether they be attenuated or amplified (Kasperson et al., 2003).
A holistic framework for assessing an individual's awareness of and preparedness for a natural hazard event, and a list of possible data items for inclusion in a tool for data collection, is provided by Enders (2001).This framework, on which questionnaires can be modelled, contains hazard knowledge, attitudes to risk, previous experience of emergencies, exposure to awareness raising, ability to mitigate/prepare/respond and demographic details.In order to provide a more holistic analysis of the emergency awareness and preparedness issue, all of the aforementioned factors must be considered (Enders, 2001).
Regardless of framework, methodological issues in relation to questionnaire development and implementation will occur.After reviewing a multitude of published research on the human dimension of disasters, Drabek (1986) stressed that methodological issues in relation to survey research (e.g.design flaws, inadequate sampling and poor measurement) must be overcome.
Practical, well developed methods of attitude measurement and psychological scaling should be applied in risk perception studies (Sjöberg, 2000).Asserting that it is not difficult to measure beliefs and attitudes about risk perception, Sjöberg (p. 409, 2000) stated: "People can be asked to make ratings of size of perceived risk on a scale, say from 0 (no risk) through a number of defined categories to a maximum risk, perhaps defined as "an extremely large risk".Such ratings have been found to be quite useful." With respect to design flaws, the careful refinement of questions is an essential part of the research journey when moving from a set of hypotheses to a theory of hazard behaviour (Kates, 1971).This is because different results are generated when question wording differs from one instrument to the next, and often in interpretation (Drabek, 1986).
The conceptual clarification of highly significant independent variables is also essential for homogenous interpretation of questions.For example, Mileti et al. (1975) questioned if "hazard experience" included false warnings or, direct observation and in-depth discussion of a hazardous event that occurred in a neighbouring community.
Broadening the scope of research on hazard experience, both Bourque et al. (1997) and King (2002) discussed the range of issues encountered in post-disaster research.While King's disaster research was based on rapid response studies (i.e.producing results a maximum of a few months after the event), Bourque et al.'s research was conducted between 217 and 712 days following various natural hazard events.Standardised questions are prominent within each study (e.g.questions concerning warnings), but additional, location specific questions are include to generate information on severity, timing, location of the event and regional issues (King, 2002).
Based on a review of 130 post-disaster studies, King (2002) concluded that longitudinal community surveys were the most constructive as they were able to produce a measurement of change in hazard awareness and preparedness over time, and in response to subsequent hazard experience.In order to record this change, the same questionnaire must be used, with only minor modifications.Bourque et al. (1997) used the replication of common questions, asked in identical or highly comparable ways, in six different questionnaires to examine how residents do or do not differ between geographic areas, time and hazard events.Furthering this analysis, they were able to ascertain the extent to which memory decay or enhancement occurs across time following an event.
When individuals are asked to self-report or self-assess their level of actual hazard awareness, experience and preparedness problems can occur.This is due to the facts that individuals may lack the ability to quantify their actual knowledge or may not be at ease in providing a truthful response (Enders, 2001).Further, participants may instinctively respond yes/no without consideration or may assess their own knowledge or behaviour on a level different to that perceived by emergency management agencies (Enders, 2001).In order to measure actual awareness, experience and preparedness, participants should be asked to describe what they know or what actual preparedness strategies they have adopted.King (2002) revealed that most post-disaster surveys consist of relatively short questionnaires administered either face-to-face or by telephone.Bourque et al. (1997) advocated the use of telephone interviewing.Although Sjöberg (2000) addressed the problem of interviewer bias, he highlighted that it is predominantly related to unstructured interviews and that interviewers conducting face-to-face or telephone interviews using a structured questionnaire have a relatively minor impact.
Despite the benefits of using structured questionnaires as a tool for generating information on the social aspects of natural hazards, research (e.g.Haynes et al., 2008a;Johnston et al., 2005) has shown that in order to capture the true complexity of a hazard in a societal context mixed methodologies which employ both qualitative and quantitative techniques should be applied.In relation to hazards research, Rohrmann (1998) notes that quantitative and qualitative approaches generate different results and as such are complementary.King (2002) reports that the questionnaire is often supported by other forms of data collection such as interviews, observations and secondary data sources (government reports, emergency management records, census databases etc).
Considering the multitude of studies conducted since 1945, it is obvious that there are many well-designed survey instruments available for generating social data in relation to natural hazards.For example, Rohrmann (2004) provides a list of questionnaires which contain risk propensity and/or risk aversion scales and presents several new questionnaire instruments for examining risk attitudes, behaviour and motivation.Consequently new structures for collecting data are not necessary.However, some form of standardised procedure is required for gathering information, assembling the data and guaranteeing worldwide accessibility to them (Alexander, 1993).It is essential that the research methodology is legitimate and replicable (King, 2002).In order to achieve this, research articles must provide a detailed account of the research methods in addition to providing access to the questionnaire.
Despite popular use of questionnaires for acquiring social data in relation to natural hazards, the techniques employed for their development and implementation have been a relatively neglected topic within published research articles dealing with hazards, risk and disaster.In line with Hawkes and Rowe's (p. 630, 2008) findings, "future studies ought to be more comprehensive in the descriptions of their methods".In order to rectify this situation, this article offers a template for researchers responsible for conducting social investigations in natural hazards research.It is not the purpose of this article to provide another best practice questionnaire or review previous instruments in terms of whether or not they can be classed as best practice.However, this article highlights the basic techniques used in developing and implementing questionnaires and it emphasises the need for researchers to provide enough detail on these important methodological features.To achieve this, a review of the social science literature on questionnaires has been provided.Based on this review, it is concluded that research articles, as a minimum, should include: -The questionnaire as an appendix or available electronically.
In doing so, this will not only allow the work to be compared and/or reproduced but it will also allow us to build on our current understanding, knowledge and practice.As a result, the natural hazards research community will benefit from reliable, replicable and valid results.
Questionnaires can be used to reveal information on public knowledge, attitude, perception, experience and preparedness levels in relation to natural hazards.When this information is combined through a mixed methods approach, robust results can be obtained, which are both comprehensive and quantifiable, adding an invaluable perspective to the development of appropriate risk mitigation and adaptation strategies.

Fig. 1 .
Fig.1.The tourist destination of Þórsmórk to the west of Mýrdalsjökull in southern Iceland.Highlighted on the map are the western jökulhlaup hazard zone, an outline of the Katla caldera and the three regional evacuation centres in the towns of Hella, Hvolsvöllur and Skógar.One four-wheel-drive road, crossing 10 glacial rivers and tributaries, gives tourists' access to Þórsmórk.Rather than crossing these fast flowing channels, tourists will be instructed to stay in Þórsmórk during a volcanic crisis.They will not be advised to evacuate to the designated centres.

---
Response format (Were open or closed questions incorporated in the design?If closed, what levels were used to measure the degrees of difference?Were participant responses limited by predefined answers?) Delivery mode (Was the questionnaire administered or self-administered?If administered, how was interviewer bias minimised?)Sampling techniques (Which probability or nonprobability technique was used to select the sample?If a probability technique was employed, how representative is the sample of the population of interest?)Response rate (How was non-response bias dealt with?)

Table 1 .
Examples of the use of the questionnaire survey instrument as a fundamental tool within natural hazard research projects.

Table 2 .
A review of 46 questionnaire based natural hazard research articles and the number of articles that provided: response format, delivery mode, sampling technique ).

Table 7 .
Advantages and disadvantages of the more common modes of questionnaire distribution within a quantitative framework (after Bird and Dominey-Howes, 2008 and references therein).

Table 8 )
(McGuirk and O'Neill, 2005;Patton, 1990)ample which allows researchers to make statistical generalisations about a wider population Use of questionnaires for acquiring information on public perception of natural hazards 1315(de Vaus, 2002).However, not all samples are representative.Some studies need to be conducted on populations whose demographic characteristics are unknown.Non-probability sampling does not allow researchers to make statistical generalisations but it is commonly used when the selection of participants is based on a known common characteristic(McGuirk and O'Neill, 2005;Patton, 1990)but population demographics are unknown.Probability sampling is typically associated with quantitative research while nonprobability sampling is associated with qualitative research.

Table 9 .
A summary of key characteristics and specific survey questions for the tourist group.All data are given as a percentage.Some sections do not equal 100% due to rounding.

Table 10 .
A summary of key characteristics and specific survey questions for the tourism employees group.All data are given as a percentage.