This content analysis study examined whether a disaster preparedness
education smartphone application created by a New Zealand city matches
what experts say are effective ways of teaching disaster preparedness
and teaching via an app. The app was assessed to ascertain if it
excelled as a platform for learning, communicated effectively,
addressed factors that can hinder people from preparing, and provided
information for special needs populations. Results show that the app
addressed most factors that hinder preparedness, provided information
for some special needs populations, excelled as a platform for
learning, and communicated effectively. The app's weaknesses include
the fact that it lacked targeted information for low-income
residents. The findings provide insight on how organizations can
effectively use apps to teach disaster preparedness.
Following a 2011 earthquake that killed 185 people and cost over USD 25 billion, disaster preparedness is a key concern in New Zealand. Auckland Civil Defence, the disaster agency of the country's largest city, has warned that only one in 10 Aucklanders is ready for disasters (Day, 2014). To address this, the agency has created a smartphone application (app) to teach residents disaster preparedness. The agency is not alone in using an app for disaster preparedness education: the U.S. Federal Emergency Management Agency and Emergency Management Australia also have apps for this purpose.
When this researcher examined the Auckland Civil Defence app in February 2015, it was organized as follows. First time app users were shown welcome screens explaining the app's purpose. After these screens, a user could log in using an email address or a Facebook username. Once logged in, the user faced four pages: home, plan, people, and info. The home page showed critical information about pending or occurring disasters as well as preparedness events such as evacuation drills.
The plan page allowed a user to create a preparedness plan in six steps that included reading about disasters that can affect Auckland, entering emergency contact information, listing emergency supplies, and indicating an emergency meeting point. On the people page, a user could connect with friends and family members via email, SMS or Facebook. With these connections, a user could see the progress peers were making in crafting disaster plans.
The info page described disasters Auckland is at risk of, showed disaster related events occurring in Auckland, explained how special needs groups should prepare for disasters, and outlined special disaster information (e.g., insurance). In addition, how to contact emergency officials, what radio stations to listen to, how to receive SMS alerts, how to volunteer, and how to access disaster help were explained. Figures 1–4 show images of the app.
This study examined the Auckland Civil Defence's app to understand two things. First, whether the app taught in a way that matched what experts say are effective ways of teaching people to prepare for disasters. Second, whether it taught in ways matching what experts say are effective ways of teaching via a mobile app. Answering these questions can help others considering using apps to teach disasters preparedness. The paper opens by describing what is known regarding effectively teaching people how to prepare for disasters and effectively teaching people via mobile devices. The research questions are outlined, the methodology is explained, findings are revealed, and implications as well limitations are discussed.
Researchers have shown that a variety of factors can help or hinder efforts to teach people disaster preparedness. These factors include environmental cues, social context, source credibility, warning channels, warning content, past experience, lack of resources, and residency barriers (Lundgren and McMakin, 2013). When people see or hear cues in the environment that show that a hazard exists, some are more likely to prepare (Sattler et al., 2000). Social context, the relationships people are part of, also influence disaster preparedness. Some people are more likely to prepare when they see friends preparing (Patterson et al., 2010). How people perceive the credibility of a source can influence preparedness actions. Because people vary significantly in regards to whom they find credible, researchers recommend using multiple sources to encourage preparedness (Sorensen and Sorensen, 2006). Preparedness messages are also distributed through conversations, television, websites, newspapers and other channels. Because people vary in regards to which channels they pay attention to and find trustworthy, researchers recommend distributing preparedness messages through diverse channels (Lindell and Perry, 2004).
In regards to message content, researchers recommend that preparedness messages should counter arguments or myths that people have regarding disasters preparedness. Messages should be repeated so that people pay attention but not excessively to avoid annoying people (Lindell and Perry, 2004). Past experience with disasters can also influence preparedness actions. Drabek (1999) noted that depending on circumstances, individuals who have experienced a disaster in the past are more likely or less likely to follow preparedness advice. Preparedness messages should therefore factor past experience.
Demographic characteristics also impact disaster preparedness. In particular, low-income individuals, recent immigrants, and individuals with disabilities face challenges preparing for disasters (Bolin, 2006). Many preparedness campaigns ask individuals to purchase extra food and supplies which can be difficult for the poor. Some communities with large immigrant populations not fluent in the local language do not provide translated disaster preparedness education material or provide poorly translated items (James et al., 2007). Undocumented migrants sometimes do not access government provided disaster services because they fear arrest. For example, during Hurricane Ike in Texas, some undocumented individuals did not seek help from government agencies because officials refused to ease immigration policing during the early part of the disaster (Wilson and Tiefenbacher, 2012). Many communities also do not adequately address challenges people with disabilities face when preparing for disasters. The result is that many with disabilities are hurt or die in disasters (Rooney and White, 2007).
This literature revealed to this researcher that one way to assess the Auckland Civil Defence app could be to examine how well it addressed factors that can hinder disaster preparedness. For example, whether the app showed users cues in the environment illustrating the likelihood of a disaster occurring or how well the app addressed myths people may have about preparing for a disaster. From this literature, the first question that guided this study was therefore developed:
RQ1: Does the Auckland Civil Defence app address these factors that can hinder people from preparing for disasters: environmental cues, social context, source credibility, warning channels, warning content, past experience, lack of resources, and residency barriers?
Across the world, there is increased realization that disaster preparedness efforts should factor in people who are marginalized and vulnerable (e.g., racial-ethnic minorities, individuals with disabilities) (Blaikie et al., 2014). In the US, the lessons learned after Hurricane Katrina have spurred efforts to ensure that disaster preparedness efforts take into consideration the challenges faced by traditionally marginalized groups (Baker and Cromier, 2014). Kailes and Enders (2007) argued that in order to avoid marginalizing vulnerable individuals, disaster preparedness education efforts should address the requirements of those with communication needs: individuals with reduced or no ability to hear, see, or speak; those with limited or no ability to speak, read, or understand the local language; and those with limited ability to understand or learn. Preparedness educators should also provide information for those with medical needs (e.g., chronic conditions), those who utilize animals or devices to maintain independence (e.g., wheelchairs, hearing aids), and those requiring transportation assistance (e.g., those who cannot afford a vehicle and those legally restricted from driving). In addition, information should be provided for care providers of individuals requiring supervision (e.g., nurses for individuals with dementia) (Kailes and Enders, 2007).
These works revealed that a second way to assess the Auckland app could be to examine how well it addressed the challenges marginalized and vulnerable individuals face when preparing for a disaster. Specifically, how well the app provided information on how people with unique communication, medical, independence, supervision, and transportation needs can prepare for a disaster. From this literature, the second question that guided this study was crafted:
RQ2: Does the Auckland Civil Defence app provide preparedness information for people with communication, medical, independence, supervision, and transportation needs?
Another issue experts say preparedness education should cover is animals. In past disasters, some people have refused to evacuate due to fear regarding what will happen to their pets or livestock. Many shelters do not accept pets while livestock pose unique evacuation challenges. Experts emphasize that preparedness education should address animal owner needs (Heath, 2001; Thompson, 2013). This literature revealed that a third way to assess the New Zealand app could be to ascertain how well it addressed issues faced by pet and livestock owners. Therefore, this third research question was drafted:
RQ3: Does the Auckland Civil Defence provide preparedness information for animal owners?
The next section examines research on how to effectively teach and communicate information using apps.
Apps, as teaching tools, are advantageous in various ways. Removed from the confines of classrooms, app users can learn from anywhere they have mobile devices (El-Hussein and Cronje, 2010). Apps also allow material to be learned and put to use immediately. For example, residents can browse a grocery store and use an app to learn what items to buy for an emergency. Learning on apps is often cheaper and brings lessons to those who find attending classrooms difficult (Gu et al., 2011).
Despite their positives, apps have limitations. Learning on an app can be fragmented because without supervision, learners on apps can stop or digress anytime. This loss of continuity can hinder learners from grasping concepts that require uninterrupted, supervised focus. Apps also often lack instructor feedback leaving learners without answers to their questions (Park, 2011). Creating apps is also a complicated process requiring knowledge in programming, mobile learning, and specialized subject matter (e.g., disasters). In addition, app designers have to make sure apps work on diverse hardware and software. Many organizations cannot meet these requirements and create apps ill suited for learning (Harrison et al., 2013).
The realization that poorly designed apps can fail drives research on how to design effective apps. Numerous organizations that have invested millions in apps want to know if their apps meet the needs of users and are achieving desired goals. Various experts on mobile learning have developed guidelines on how to design mobile learning platforms such as apps that are usable and excel as platforms for learning. These experts (e.g., Dillard, 2012; Elias, 2011; Kukulska-Hulme and Traxler, 2013; Wang and Shen, 2012) have argued that mobile learning platforms such as apps, in order to excel as platforms for learning, should: (a) have intuitive and simple interfaces, (b) incorporate multimedia because people find learning easier if they see more than just text, (c) present lessons in short chunks to accommodate the fact that human short term memory is limited, (d) be engaging and entertaining, (e) provide content that is relevant to learners, (f) deliver information fitting users' immediate needs and contexts, and (g) provide access to information even when not connected to wireless networks. From this review, it was evident that a fourth way to assess the Auckland app could be to ascertain if the app possessed characteristics that scholars argue are present in apps that excel as platforms for learning. Hence, this fourth research question was drafted:
RQ4: Does the Auckland Civil Defence app exhibit principles of an app that excels as a platform for learning: (a) simple and intuitive interface; (b) incorporates multimedia; (c) information that is contextual and relevant to the learner; (d) just-in time delivery; and (e) offline access?
Additional guidelines on designing apps that excel as learning platforms have emerged from health research. As health related apps have grown in numbers, scholars worried about the accuracy of the information in these apps have developed principles on how to design apps that communicate effectively. An example is the principles developed by Huckvale et al. (2012, 2015) in their research on how to design asthma apps that communicate effectively. These researchers have argued that an app that communicates effectively should have information attributed to an author whose qualifications are mentioned (when information was written and modified should also be indicated), claims supported by evidence, and a privacy policy regarding how app user information is used. In addition, who paid to create the app should be identified, external influences that could affect the objectivity of app content should be revealed, and a way to contact app creators should be provided. The purpose of the app as well as the intended audience should be clearly explained and a disclaimer should be presented that the app is not meant to replace advice from human professionals (Huckvale et al., 2012, 2015). This literature revealed that a fifth way to assess the Auckland app could be to assess if the app possessed the characteristics that scholars have argued are present in apps that communicate effectively. This fifth research question was therefore drafted:
RQ5: Does the Auckland Civil Defence app exhibit principles of an app that communicates effectively: (a) information attributed to qualified author, (b) app purpose explained, audience identified, creator identified, and disclaimer present, (c) privacy policy present, (d) content creation and modification date present, (e) claims supported by evidence, (g) who paid for app identified, (h) external influence that could affect objectivity of content identified, and (i) contact information of app creators provided?
Researchers seeking to evaluate apps have a variety of methods at their disposal. These include conducting experiments, watching people using an app in the real world, and surveying users to get their input (Nayebi et al., 2012). In this study, this researcher decided that an appropriate method for evaluating the Auckland Civil Defence app would be one in which two individuals of similar educational and professional backgrounds would utilize the Auckland Defence app and afterwards, use a checklist to assess whether the app met the characteristics outlined in the research questions. This checklist approach mirrors the app assessment method utilized in past studies that have evaluated apps such as Reynoldson et al. (2014), Bender et al. (2013), and Huckvale et al. (2012, 2015).
This checklist method has several limitations that were highlighted by Alshehri and Freeman (2012) who explained that apps have several characteristics that make it difficult to assess them. In everyday life, apps are used in various scenarios (e.g., while running, driving) by numerous people (e.g., technology experts, novices) using numerous devices (e.g., expensive devices with newest software, older devices) on different wireless networks (e.g. Wi-Fi, 4G, 3G,). These differences make it hard to assess apps because in some cases, it is not clear if the problem is with the app, the hardware of the mobile device, the software running the mobile device, the wireless network, or the person using the app. To address these potential pitfalls, this researcher ensured that those evaluating the Auckland Civil Defence app, in addition to being of similar educational and professional background, would assess the app on identical devices using a common wireless network while carrying out uniform tasks.
The first step in crafting the app analysis method was to develop a checklist for ascertaining whether the app: (a) had a simple and intuitive interface, (b) incorporated multimedia, (c) had information that was relevant to the learner, (d) offered just-in time delivery, and (e) offered offline access to data. In order to craft such a checklist, this researcher had to figure out how to measure whether the app was simple and intuitive. To find such measures, studies of simplicity and intuitiveness in other digital media were examined. What emerged was that an influential way of determining simplicity and intuitiveness is Taylor and Kent's (2014) dialogic framework. These scholars argued that simple and intuitive digital media have a guide or map that tells users where to go in the media, a search box that can be used to find out what is in the media, a navigation indicator showing the part of the media the user is on, and a button or link that easily takes a user back to the homepage of media. In addition, the media should lack animation that can strain processing power and pages should load in less than 0.1 s (Kim et al., 2014; Lee, 2014). These items were adopted as measures for assessing whether the app was simple and intuitive. They were added to the other items (incorporates multimedia; has information relevant to the learner; offers just-in time delivery; offline access to information) to create a checklist for assessing the app.
A second checklist this researcher created was one for determining whether the app matched the principles of an app that communicates effectively. The principles utilized by Huckvale et al. (2012, 2015) to assess asthma education apps were used to develop this checklist. A third checklist was drafted to determine whether the app addressed factors that can hinder people from taking steps to prepare for disasters. Lindell and Perry's (2012) Protective Action Decision Model, which details what factors can prevent people from preparing for a disaster, was utilized to draft this checklist. A fourth checklist was created to assess whether the app provided information for people with communication, medical, independence, supervision, and transportation needs as well as animal ownership needs. Kailes and Enders' (2007) Function-Based Framework for Emergency Management and Planning which describes how to best prepare individuals with special needs for disasters was utilized to draft this checklist. These checklists can be seen in Tables 1–4. Data collection occurred 1–10 February 2015. The app was downloaded to two identical smartphones running the latest version of the Apple iOS operating system. The smartphones did not contain any additional apps except those installed at the factory. Two independent coders (both PhDs in mass communication and employed as college professors at universities in the United States) randomly selected four sections of the app and analysed them using the above-mentioned checklists. This analysis was conducted while the smartphones were connected to the same WIFI network in the same room at a uniform time. After this initial analysis, the coders met to discuss their findings. Initial inter-rater agreement was 95 %. Discrepancies were resolved and the coders reviewed remaining sections of the app. After completing the analysis, ratings were compared and final ratings reached by consensus.
Disaster preparedness messages are more likely to be effective if people are shown cues in the environment illustrating the likelihood of disasters occurring (Lindell and Perry, 2012). The app addressed this factor with an activity section that showed the prevalence of storms and floods in the Auckland area. This illustration of risks Auckland faces can be viewed as an example of pointing to environmental cues. Research has also shown that some people are likely to prepare if they see peers preparing (Dash and Gladwin, 2007). The app addressed this factor by allowing users to connect with friends via Facebook, SMS, or email. Through the app, users could see preparedness steps their peers were taking and thus be encouraged to also prepare.
Because people vary in regards to whom they find credible and what communication channel they utilize frequently, experts recommend that preparedness messages be distributed by multiple sources on various channels (Guion et al., 2007). The app provided a multiplicity of sources from where to obtain preparedness information. For example, the section on fire provided a link to the New Zealand Fire Service and the section on pandemics provided a link to the Ministry of Health. In addition, the app provided links on how to obtain information through radio, SMS, email, phone, and other channels.
Preparedness messages should address arguments people may have against
preparing as well as past experience individuals have had with
disasters (Sharma and Patt, 2012). The app addressed these factors by
presenting statistics. Parts of the app went into detail describing
past disasters that have affected Auckland and the likelihood of these
disasters occurring in the future. For example, the section on
aviation disasters explained that between 1996–1999, there were 40
aircraft accidents in the Auckland area. The section on cyclones noted
that every year, a cyclone comes within 500
The app did not address how low-income individuals can obtain assistance in preparing for disaster. For example, information was not provided on how the poor can obtain help purchasing items for emergency kits. The app also did not address any concerns that undocumented individuals may have about accessing disaster preparedness help. Experts recommend that disaster managers make it clear what disaster resources are available to the undocumented so that these individuals are not apprehensive about seeking official help (Santos-Hernandez, 2006). See Table 1 for complete findings in regards to RQ1.
Kailes and Enders (2007) recommended that in order to avoid marginalizing vulnerable individuals, disaster preparedness education efforts should meet communication, medical, independence, supervision, and transportation needs. The app's section on “people with disabilities or special requirements” provided information for people with hearing impairment, asthma/respiratory problems, sight impairment, and physical disability/mobility impairment. The advice it provided included organizing a support network of people who can assist with evacuation, preparing a seven-day supply of essential medication, and knowing where to access life sustaining equipment.
Where the app was lacking was in providing information for those with limited or no ability to speak, read or understand English. New Zealand is a multilingual country. Tan (2009) explained that New Zealand's government, recognizing that English fluency is a problem for many, provides a Language Line service that provides translators fluent in 44 languages to assist people seeking help from hospitals, police, charities, and other vital services. The app did not indicate how those not proficient in English could obtain information. See Table 2 for complete findings in regards to this research question.
For disaster preparedness efforts to be successful, the needs of animal owners should be included in planning (Thompson, 2013). The app fulfilled these criteria in its special section on pets and livestock. For pets, the app advised users to microchip pets, prepare emergency food, and identify pet-friendly hotels. For livestock, the app instructed owners to work with local authorities on developing measures to secure livestock in a disaster.
Taylor and Kent (2014) argued that digital platforms that excel as platforms for learning are characterized by a simple and intuitive interface. Aspects of the Auckland app that exemplified this characteristic include the presence of a navigation bar telling users where to go on the app and absence of animation that can strain processor power. The app also met other criteria outlined by Taylor and Kent such as incorporating images, providing contextual information by allowing users to receive customized disaster alerts, and providing just-in-time delivery of information by allowing the delivery of notifications when a disaster was occurring. When offline, the app did show information from the previous time it was used. The app was lacking only in one area: it lacked a search box where a user could enter text to search to find specific items. See Table 3 for complete findings in regards to this research question.
According to Huckvale et al. (2012, 2015), an app that communicates effectively must provide full contact information of the app's creators. The Auckland app met these criteria by providing the email and phone contact information of the Auckland Defence Council. Additional criteria that the app met include describing the full purpose of the app and who created it. The app noted, for example, that the Auckland Civil Defence commissioned the app, a company called CloudM created the app software, and a company called Ogilvy and Mather designed the app interface. The app also justified claims by providing statistics, was transparent in revealing that it was funded by the Auckland government and it contained a privacy policy regarding user data. Pages of the app with detailed information also had notes indicating when they were last updated.
Where the app did not meet Huckvale et al.'s (2012, 2015) criteria was in two areas. First, the information was not authoritative. At the bottom of each update, the name, title, and contact information of the person who wrote it were not provided. People, in many cases, find information more credible if an actual author is listed (Cheever and Rokkum, 2015). A second weakness was the lack of a disclaimer that the app was not supposed to replace human advice. Apps, like most technology, are prone to errors and especially in disaster settings, can malfunction. Some apps include a disclaimer they are not supposed to substitute advice from human experts (Boulos et al., 2014). See Table 4 for complete findings in regards to this research question.
The findings contain valuable lessons for organizations around the world seeking to use apps for disaster preparedness education. The first lesson is that it is possible to create an app that addresses factors that can hinder people from taking steps to address disaster. Organizations can accomplish this by creating apps that show disaster cues from the environment, allow users to connect with friends or family who are preparing for disaster, provide information from multiple sources, show users how to obtain information from alternative channels, address arguments people may have against preparing for disasters, and address past experiences users may have had with disasters. Additional lessons from the Auckland app is that it is possible to create an app that excels as a platform for learning, communicates effectively, and provides information for people with unique medical, independence, supervision, transportation, and pet/livestock needs.
The findings from this app analysis, however, also reveal several things the Auckland app failed to achieve that other app creators worldwide should be cognizant about. First, app creators should recognize the need to provide information on how low-income people can address challenges to preparing for disaster. Telling people to prepare but not providing them information on how they can address lack of resources (e.g., where to get help in stockpiling food if you lack money) is problematic. In Auckland, an estimated 70 food banks serve the underprivileged (Jackson, 2013). These statistics mirror other cities in emerging and developed countries where poverty is a part of urban life for many. Disaster preparedness app creators can make their apps stronger by including information on how the underprivileged can get help with disaster preparedness.
Second, app creators should also be aware of the need to provide information to people who do not speak, read, or understand the local language. In Auckland, police, hospital and other emergency services utilize a Language Line service to provide assistance to the 44 different nationalities in New Zealand that are not fluent in English (Tan, 2009). The Auckland app could have been stronger if it had contained information on how individuals not fluent in English could access this service. In other regions, app designers should factor how they will address the linguistic diversity among users.
App creators should also address challenges undocumented migrants face preparing for disasters. New Zealand, like many nations, has a growing population of migrants. Glass (2013) noted that some of these individuals in New Zealand, documented and undocumented, live in fear of deportation and are often exploited by employers who pay them below the minimum wage knowing that they are afraid of going to government authorities to complain. App designers can make their apps stronger if they include information clarifying to undocumented individuals what preparedness services they can access regardless of immigration status. Lastly, app creators should consider including a search box, adding author names to articles, and adding a disclaimer that the information in an app is not meant to replace human professional advice. The presence of these elements can make apps more effective.
This study has several limitations that future scholars need to address in order to enhance the existing body of knowledge regarding the use of apps as tools for teaching disaster preparedness. First, this study lacks insight from users regarding what they think about the app. Surveying and interviewing people who used the Auckland app could have provided significant insight into whether the app is succeeding in preparing Aucklanders for future disasters. In addition, querying users could have shed light on whether Auckland Civil Defence is using the app not simply to talk at people but to give them a voice in shaping how disasters are managed. A second limitation of the study is that it did not explore whether the app is entertaining or engaging. This is something that can also be ascertained by interviewing or surveying users. Smartphone users have a multiplicity of apps to choose from and if an app is not engaging or entertaining, it is likely they will not use it. Whether the Auckland app is entertaining and engaging should be understood before other organizations imitate the app's approach to preparedness education. A third limitation of the study is that it did not explore whether the app delivered information in short chunks appropriate for mobile learning. The Auckland app has lots of information but is it too much or just the right size for users? Surveys and interviews can also shed light on this and give valuable details on exactly how much text and images developers should put in preparedness apps.
Does app address factors that can hinder preparedness?
Does app provide preparedness information for people with communication, medical, independence, supervision, transportation, and pet-livestock needs?
Does app excel as platform for learning?
Does app communicate effectively?
Welcome screens of the app.
The home page of the app.
The plan page of the app.
The info page of the app.