Analysis of a risk prevention document using dependability techniques: a first step towards an effectiveness model

Major hazard prevention is a main challenge given that it is specifically based on information communicated to the public. In France, preventive information is notably provided by way of local regulatory documents. Unfortunately, the law requires only few specifications concerning their content; therefore one can question the impact on the general population relative to the way the document is concretely created. Ergo, the purpose of our work is to propose an analytical methodology to evaluate preventive risk communication document effectiveness. The methodology is based on dependability approaches and is applied in this paper to the Document d'Information Communal sur les Risques Majeurs (DICRIM; in English, Municipal Information Document on Major Risks). DICRIM has to be made by mayors and addressed to the public to provide information on major hazards affecting their municipalities. An analysis of law compliance of the document is carried out thanks to the identification of regulatory detection elements. These are applied to a database of 30 DICRIMs. This analysis leads to a discussion on points such as usefulness of the missing elements. External and internal function analysis permits the identification of the form and content requirements and service and technical functions of the document and its components (here its sections). Their results are used to carry out an FMEA (failure modes and effects analysis), which allows us to define the failure and to identify detection elements. This permits the evaluation of the effectiveness of form and content of each components of the document. The outputs are validated by experts from the different fields investigated. Those results are obtained to build, in future works, a decision support model for the municipality (or specialised consulting firms) in charge of drawing up documents.

1 Introduction

Every year, major natural phenomena cause human and material disasters. Recently, in August 2016, an earthquake of magnitude 6.2 occurred in central Italy causing 250 deaths (Guha-Sapir et al., 2016). Two months before, in the same year, the River Seine in France rose to a height of 6.10 m and overflowed, causing four deaths, 24 injuries and a great amount of material damage throughout the different departments (France territorial divisions) it traverses (CCR, 2016). Preventive policies have been implemented to manage the consequences of these disasters, such as the Hyogo or Sendai frameworks for action and disaster risk reduction (UNISDR, 2015). Transmission of preventive information is equally important and has been the topic of current discussions highlighted by recent scientific research (Newell et al., 2015) and international institutions as the United Nations (UNISDR, 2015; United Nations, 2006).

Figure 1 A prefect is a government representative of an area or department. It is thus responsible for public order, ensures the application of laws and regulations and verifies that the local authorities (town hall or EPCI, which is a grouping of town halls) respect them as well. French regulatory prevention information throughout the town include the Territorial Coherence Scheme (SCoT), Knowledge of Existing Data (PAC), Flood Risk Prevention Plan (PPRI), Development and Planning Guidelines (OAP), Local Urban Plan (PLU), Departmental Document on Major Risks (DDRM), Municipal Information Document on Major Risks (DICRIM), public notices, Purchaser Tenant Information (IAL), Family Plan for Safety Layout (PFMS), public meetings, floodmarks.

In France, numerous prevention systems and organisations exist to manage both natural and technological hazards. Risk prevention requires the involvement of many stakeholders ranging from public authorities, experts and infrastructure managers to individuals and communities. In France, the legislation that relates policies and prevention of natural hazards is enforced through different legislation. In 1987, the need for emergency management was addressed and the right of access to preventive public information has been recognised. In 2004, the 1987 law was updated and modernised. Notably, the changes demanded disseminating preventive information to the general public (Observatoire Régional des Risques Majeurs en Provence-Alpes-Côte d'Azur, 2017). An example is demonstrated in Fig. 1 which shows the regulatory tools linked to urban development and/or used for disseminating preventive information and its exchange between different stakeholders. For instance, in a town threatened by floods, the mayor must make a Flood Risk Prevention Plan (PPRI) that will be approved by the prefect. The general public will then give their opinion on this plan. Different forms of communication (public meetings, information kits, posters, etc.) are used not only to provide safety recommendations, but also as means of increasing individual knowledge of risk.

This is highly significant as human behaviour during major disasters is influenced by their own knowledge of risk (Quarantelli, 2008). Behaviours, however, are not necessarily the result of logical and rational reasoning. Attitudes of denial or risk underestimation, often found among inhabitants exposed to natural hazards, are commonly interpreted as an “addiction to risk”, and sometimes judged as an incomprehensible unawareness in the face of a known and visible danger. This is the cognitive dissonance phenomenon described by Festinger (1957). For example, a person living in an area that they know to be exposed to a natural hazard is clearly in cognitive dissonance (Schoeneich and Busset-Henchoz, 1998). This makes it possible to better understand certain attitudes of scepticism, or even mistrust, towards technical or scientific studies and information. Indeed, to reduce the psychological discomfort caused by dissonance, individuals can act on their risk representations. Cognitive biases can then substitute knowledge of individual risk as an act of denial, which consists of trivialising the risk involved and is similar to a phenomenon of disbelief (Weiss et al., 2006) or an illusion of control, which consists of believing that one has control or influence over external or random events (Langer, 1975). Kahneman and Tversky (1979) and then Slovic et al. (1982) argue that people form their judgments heuristically, based on their experience, habits or cultural traditions that enable them to construct their cognitive risk representation. In fact, people fear that they will find themselves at risk and that their certainties will be undermined by reviewing information on the risks affecting their living environment.

Residents are also generally thought to have a lower human vulnerability than tourists, as risk awareness is often attributed to locally rooted populations (Hubert and De Vanssay, 2005). However, in the sample of victims of floods in the Mediterranean between 1998 and 2011 studied by Boissier (2013), only 30 out of 203 people were non-residents. Even though several other factors may have to be considered, non-residents are less vulnerable because they are more prone to taking instructions, more respectful of evacuation orders and less inclined to take risks (going to school to pick up their children, saving personal belongings because less property needs to be saved, etc.) as pointed out by Ruin (2007).

However, communication on risk can become a form of indirect experience of risk and thus a way of strengthening its acceptance and stimulating the involvement of exposed populations (Festinger, 1957). If information is transmitted effectively people are more likely to adopt the relevant behaviour during the event as they have better knowledge of the associated risks and the safety recommendations for better risk prevention (Siegrist and Cvetkovich, 2000). Results from a research study showed that information seeking by residents seems to coincide with the intention to take preventive actions (Kievik and Gutteling, 2011). Providing regulatory information to residents can also lead to further information seeking (Hagemeier-Klose and Wagner, 2009). Consequently the public needs to be periodically informed about the hazards and the levels of risk they are exposed to and how their situation is changing (United Nations, 2006). For Paton et al. (2000) a major challenge for risk communication is ensuring that the information provided is meaningful to recipients and motivates risk acceptance.

Although significant progress has been made (recent decrees, investment by towns, more resources, etc.), certain features regarding risk information remain disappointing such as the uneven implementation and lack of control over these resources (IRMa and al., 2015), different behavioural instructions from one document to another even in reference to the same phenomenon (MAAF and MEDDE, 2013), incomplete information about the hazards to which the population is exposed to and so on. It is also very difficult to establish if preventive communication tools achieve their aim in terms of being appropriated by the local population (AFCPN, 2013). Do they empower residents with respect to knowledge on the risks that concern them? (Cutter, 1993; Lindell and Perry, 2004). To give answers it is important to analyse the effectiveness of such preventive communication tools. Effectiveness is the level of achievement of planned activities and achievement of expected results (Association Française de Normalisation, 2005). It refers to the concepts of function and dysfunction. Are those preventive tools made to be effective in terms of appropriation by the general public? It is important to evaluate if those preventive communication tools reach their goal while conforming to the law.

To rise to this challenge, this article has the objective of proposing an approach capable of analysing preventive communication document, in terms of its effectiveness. The Municipal Information Document on Major Risks (DICRIM) is used as an application of the method proposed. The article is structured according to the following plan: it first examines selected methods for analysing the effectiveness of communication tools, and it then presents the methodology proposed after which the results of its application to the DICRIM are shown. A part of these results is applied to a DICRIM database. The article ends with the conclusion and outlook.

2 State of the art

2.1 Effectiveness of risk prevention communication tools: case of the DICRIM

In this work, we focus on preventive risk communication in regulatory documents (informal communication by social networks and by word of mouth is not considered). In France, information and knowledge are communicated at a communal level using different means (see Fig. 1). The mayor establishes DICRIM intended for the public and based on information contained in DDRM (Departmental Document on Major Risks) made by the prefect. DDRM lists all essential information on major natural and technological risks at the level of their department. The municipalities required to produce a DICRIM are those in which there are Flood Risk Prevention Plans and/or those designated by prefectural decree because of their exposure to a particular major risk.

The mayor must also inform the population about major hazards at least every 2 years by way of public meetings. In addition, the safety instructions contained in the DICRIM and those which may be laid down by the operators or owners of the premises and land referred to in Article R. 125-14 of the Environmental Code are made public by means of posters. These posters are made based on a template set by the state. The mayor is also required to establish floodmarks in the municipality and carry out an inventory. They correspond to historical or exceptional new floods. They make it possible to refine the expertise of historical floods, but also to bring to life the memory of flood risk. With regard to residents, there are two distinct obligations regarding preventive information established by the Environmental Code for purchasers and tenants: (i) information on the natural hazards affecting the property and (ii) information on claims resulting from recognised natural disasters which affected all or part of the property concerned. This Purchaser Tenant Information (IAL) concerns: all lessors or sellers and all types of real estate, regardless of the destination (seasonal rentals also; MEDDE, 2013). The general public is also encouraged to establish their PFMS (family safety plan). PFMS is a booklet, provided by the state, which contains sections to filled out with relevant information by members of the public. For instance, the booklet helps them to check if they own all equipment needed in the event of a crisis.

We chose to focus on DICRIM because, in the French government's overall strategy for risk prevention, it is the main reference document in terms of informing the public about the natural and technological risks affecting the municipal territory. A brochure, film or advertisement may be just one part of a larger campaign to promote hazard readiness, and these items can and should be tested separately for efficacy and effectiveness prior to broad deployment (Sanquini et al., 2016).

In general, inequality in terms of updating the specifications, transmitting the information and controlling the execution of preventive information tools is observed and this concerns also the DICRIM. For example, in the Provence-Alpes-Côte-d'Azur (PACA) region, in December 2016 only half of the concerned communities had completed their DICRIM (ORRM, 2017). Furthermore, the French government gives indications only on the general content of the document without providing any standards notably regarding the design of the document. Works and writing guides have been proposed to develop DICRIMs (Clément et al., 2012; CEPRI, 2013; IRMa, 2014) but the justifications of these recommendations are not clearly and fully presented, which leads to disparate documents in terms of content and form. Given the large number of pages, the effectiveness of some DICRIM for residents is also questionable, as is that of other DICRIMs containing a reduced number of pages, which can lead to oversimplification and significant loss of information. Finally, there is no link between quantity and compliance of DICRIM (Douvinet et al., 2013); having a DICRIM does not mean that municipalities provide good preventive information under the law (Rode, 2012).

Effectiveness of some contents in DICRIM is also questionable. For instance, the law required that DICRIMs include maps regarding each hazard threatening the given town. Those maps are used to show to the population the hazard level geographically distributed on the territory of the municipality. But those maps are often similar to those elaborated in PPRI, which is mainly used by experts, and they are not intended to be used for communicating with the general public. Flood maps are frequently seen as an information tool rather than a communication tool (Meyer et al., 2012). That is why some maps are really difficult for some people to understand, which is regrettable because it is maps which often first draw people's attention before text. Some works propose an evaluation of flood maps effectiveness notably based on interviews or eye-tracking and also give recommendations that should be used for hazard maps contained in DICRIM (Meyer et al., 2012; Fuchs et al., 2009).

These elements must be considered carefully because individual cognitive perception is reinforced or weakened by the form and content of the provided informative document; document communication is intended to be helpful and to result in the expected impact (Terpstra et al., 2009). Consequently, assessing the effectiveness (form and content) of such legislative texts and the information they contain is an indispensable challenge, especially since this issue has received very little attention and only a few works on it have been published to date (Gominet, 2007; Kellens et al., 2013).

2.2 Analysis of candidate methodologies

Evaluating effectiveness requires the development and improvement of tools and methods. Such methods currently exist in different fields of study: advertising, geography, engineering, etc. They apply to different systems and are generally divided into three categories, according to if they (i)  involve formal population samples, (ii) do not involve formal population samples but rely on population perception assessments and (iii)  are based on systemic analytical methods that do not involve the population.

Regarding communication and marketing, advertising is one of the major tools used by companies to disseminate “persuasive” information on their markets, to inform, persuade and remind (McArthur and Griffin, 1997). Methods have been developed to assess the effectiveness of advertising (pre- and post-advertising tests). These different methodological approaches based on questionnaires are of great interest. Current studies performed to evaluate the effectiveness of preventive risk communication (including DICRIM) for the public are usually conducted using the same methodological tools: surveys (either before or after dissemination of the document) by questionnaires. Gominet (2010) and Duaut and Luneau (2008), for example, obtained their results through prior surveys of mountain communities subject to natural hazards. In the same vein, the French Minister of Agriculture conducted a series of interviews with various stakeholders on subjects including risk, to measure the effectiveness of DICRIMs or to at least obtain their views on these documents (MAAF and MEDDE, 2013). Moreover, communicating on risk can be considered as pertaining to “advertising” on how to protect oneself. As stated above, the challenge is to make the public aware of risk, get them to accept it and protect themselves by taking appropriate actions (Renn, 2014). The risk communication documents must therefore have a persuasive effect on the population in the same way as advertising seeks to convince a consumer. If the questionnaires are well constructed and the results interpreted rigourously, these methods allow for predicting the impact communication should have with regard to the pre-test and make it possible to modify the content prior to publication if necessary. In the post-test phase, they also allow for obtaining feedback on communication, on what worked or not, and on any dysfunctions in order to consider possible corrections and a new communication campaign. However, these methods also have limitations. Indeed, they require building a representative sample of the population that may require a budget and human and material resources which can be substantial. Moreover, these methods are focused on opinions and do not take into account factors that omit individuals' feelings.

Some methods do not require the use of a representative sample of the population but need indirect intervention:

• advertising efficiency standards for establishing performance comparisons, without it being necessary to use population samples;

• “connected” methods to evaluate the effectiveness of advertising such as the association of flyers, websites and/or mobile applications (QR code, website audience, image recognition, etc.). For instance, a flyer can be effective if it has a QR code. It is then possible to quantify the number of “flashes” generated by people with their smartphones to visit the associated websites. The higher the number of visits, the more effective the flyer is deemed to be;

• ROI detection elements (return on investment; click rates, visits, cookies, etc.) related to digital advertising and tracking capabilities. They allow, for example, making post-tests a certain time after the broadcast of a message to identify individuals previously exposed to it.

Connected methods can be useful in that they will help to determine whether a risk communication document aroused the interest of people by convincing them, for example, to go to the associated websites (using QR code) for additional information after reading the document. These evaluation methods also appear relevant to our case study mainly because they do not require the intervention of inhabitants to assess effectiveness. However, there is an indirect need for some people to be involved by clicking, using a QR code, etc. If it can give an idea of the effectiveness of risk communication documents, it does not provide information about the impact of its reading. Is the information really received? Although relevant in some aspects, these methods can be expensive.

Finally, the use of systemic analytical methods based on structural, functional and dysfunctional analysis of the system under study seems relevant for the analysis. Their results could be used in the next works to build an effectiveness evaluation model based on aggregated formalised indicators. Constructing an effective model will, of course, require involving the public but this will not be necessary when municipalities for instance come to use the model. In such a sense, it could provide financial gain for them. Effectiveness is the level of achievement of planned activities and achievement of expected results (Association Française de Normalisation, 2005). It refers to the concepts of function and dysfunction. Thus analytical methods, such as dependability analysis used in engineering, seem relevant. DICRIMs, and other risk communication documents, are subject to complex processes involving interdisciplinary concepts (information processing, cognition, communication, etc.). The lack of a formal, detailed framework (form and content), and lack of familiarity by the public make their effectiveness difficult to assess without a systemic and rational approach. Dependability methods allow the identification of risks and analysis of behaviour and failures. These methods are qualitative (preliminary analysis of hazards; failure modes and effects analysis, FMEA; summarised breakdowns combinations method) and quantitative (fault tree method, event tree method, etc.), based on the construction of state graphs (space states method, stochastic Petri nets), and on simulation (Monte Carlo simulation). They were developed for complex industrial systems and applied in different domains (Peyras et al., 2006; Zouakia et al., 1999; Bambara et al., 2015). These methods are interesting as they enable identifying the elements involved in the effectiveness of the system under study, determining the causes and effects of dysfunctions and listing detection elements able to pinpoint the occurrence of dysfunctions. Our work takes this direction and aims it at applying a function and dysfunction analysis method to a risk communication document. The problem to overcome is that the methods chosen must be adapted to our context, which is not one of engineering, but one of risk prevention. The first successful application of dependability methods in such a context was carried out in a field that interests us: the diagnosis of contingency plan failures (Girard, 2014; Piatyszek and Karagiannis, 2012; Bambara et al., 2015). In other words, the systemic analysis of the risk communication documents such as DICRIM can provide the means for performing qualitative analysis of effectiveness. Previous work evaluating the effectiveness of a DICRIM is based on ad hoc surveys carried out at the level of given municipalities or territories, on a given document. They also need significant human and material resources. This work differs from previous work because it allows us to obtain a generic approach based on systemic and analytical reasoning applicable to any risk communication document.

Figure 2 Global methodology proposed and followed to build the effectiveness evaluation model: detailed steps leading to the detection elements. Dotted boxes show future works.

3 Methodology proposed to assess effectiveness of a risk preventive communication document

The development of our method of analysis leads to various types of indicators (compliance, form, content), the combination of which are used to assess the effectiveness of the document. This method is generic and applicable to different prevention documents. Figure 2 shows the global approach of the methodology we propose. In this paper, results of steps shown in solid line boxes are presented. Shown in the dotted lines are potential steps.

The method proposed considers functioning and non-functioning of the system, which here is the preventive communication document.

It consists of a functional analysis that fully describes the functions and relationships in the system and the requirements it fulfils. A constraint is a “characteristic, effect or design provision that has been made mandatory or prohibited for any reason whatsoever. No other possibility is left” (AFNOR, 2015).

Once the system is analysed, a change of the granularity of analysis is made to study the system components. Structural analysis allows us to determine those components. Three types of requirements are identified linked to those components. Regulatory requirements to evaluate conformity and form and content requirements to evaluate performance are identified.

An analysis of regulatory requirements is carried out. To analyse effectiveness of the preventive communication document, first we must analyse its compliance with the law. This conformity analysis is made by identifying detection elements and indicators in future works. A database of documents is made and used to apply those regulatory detection elements.

In parallel, functions of components are analysed. Functions are systematically characterised, classified and evaluated (AFNOR, 2014). For the functional analysis we chose to use the APTE method that can be applied successfully to our case study as a quick test, as proposed in Ghariani et al. (2014). It is one of the most used dependability methods, and it generally provides the basis for a subsequent FMEA which is the next part of the approach. FMEA is an inductive method of analysing potential failures in a system. It systematically considers each system component and its failure modes one after another. Failures are identified by non-compliance with requirements. Detection elements are also listed, allowing the identification of this non-compliance. The results of FMEA analyses are presented with a diagram in a bow tie form, specifically designed for the type of system studied. The detection elements were formulated by making use of the literature. For example, the field of advertising was investigated for the part concerning the information on the preventive document's form. FMEA results were validated by experts. When performing an FMEA in an industrial situation, specialists in different areas are usually involved in completing and validating the results obtained. Several domains are necessary to evaluate the effectiveness of preventive from risk and communication, justifying the choice of experts made in our work. Three experts were asked to carry out this task: two have performed research in technological or natural risk analysis for more than 15 years, while the other is a communication expert with 27 years of experience in the field.

Later, detection elements will be formalised as indicators. An indicator is information that helps a stakeholder, an individual or a community in general, to carry out the course of actions needed to achieve a goal or to evaluate a result (Bonnefous and Courtois 2001). It should be formalised in order to make its use repeatable and reproducible (Curt et al., 2010). Indicators will be aggregated with each other to form a model that will give an effectiveness score as output. Feedback will also be provided by the model to know which component must be improved and how. This effectiveness evaluation model will be a decision support model for the municipality (or specialised consulting firms) in charge of drawing up documents. This model will allow evaluating the effectiveness of existing preventive communication documents and identifying the corrective actions needed to improve their effectiveness.

The results shown below stem from the application of the method to the case of the DICRIM. As said before, Indicators and the construction of an effectiveness evaluation model represent our future work for which only some examples will be shown in this paper.

4 System functions: external functional analysis

The external functional analysis (EFA) consists of three main steps: system definition, expression of needs and determination of relationships with external environments.

4.1 System definition and needs satisfied by the system: the DICRIM

First, it is necessary to define the system to be studied precisely. Limits are defined, which then lead to considering the interactions with external environments. The system therefore takes into account the document (DICRIM) with its content, form and accessibility as described by the relevant law.

The main headings and information to be included in the DICRIM are listed in the National Model for the application of the Environment Code (Articles L 125-2 and R 125-9 to R 125-27) issued by the MTES (Ministry of Ecological and Inclusive Transition; MEDDE, 2013). Our analysis will be based on it. It is a framework with general information that must be used both by mayor and prefect to create respectively DICRIM and DDRM. To do so they have to use this framework and to add specific information, for instance information regarding the municipal territory for mayor. The major risks that must be dealt with in the DICRIM (according to their occurrence in the given town) are listed in the National Model and are as follows: floods, earthquakes, ground movements, forest fires, avalanches, storms/cyclones, volcanic eruptions, dam failures, nuclear accidents, industrial accidents and mining risks. There is no legal obligation regarding the document's format. The DICRIM can be viewed by the public at the town hall as a paper document. In some towns it is also distributed directly to residents' mailboxes or posted on the municipal website in digitised form or more rarely, in interactive format.

EFA is used for translating the need satisfied by the system. In general, this can be formalised into three questions applied here to the DICRIM: (1) to whom is the system dedicated? To the general public; (2) on what does it act? On its knowledge and risk perception; (3) why is this action necessary? (that is to say, for what purpose does the system exist?) To provide notice on hazards and on how to act when phenomena occur.

Figure 3 Functional block diagram for DICRIM.

4.2 External circles

It is then necessary to determine the external circles that interact with the system. They are made up of human elements, natural elements or part of other systems which can act on the document or be subject to its actions. The inventory of these external circles (designer, director, regulations, residential accommodation, etc.) is established by examining the environment of the DICRIM. We also highlight interactions between these areas and the DICRIM. Figure 3 illustrates the design, construction and operation process (reading and use by the public). Interactions between the system and its environment are visualised using a functional block diagram, in which we distinguish between contact relationships (represented by straight segments) and flows (represented by arcs). Interactions between an external circle and DICRIM identify requirements to be considered. Flow relationships between two external circles identify the service functions performed by the system. The quality of the external circles and the analysis of interactions determine the completeness of the functions obtained.

4.3 Service function and requirements

Service functions reflect the objective of a system. They are determined through relations between two external environments. Figure 3 shows that the transmitter (developer and designer) is linked with the receiver through the document. This is the process involved in preventive information on hazards. With the DICRIM, the receiver, that is to say the public, is also related to major events when they happen. They must implement what they learn from the document in terms of behaviour. Two DICRIM Service functions are then deduced:

1. Service Function 1 (SF1): the DICRIM provides information on maintaining and nurturing risk culture (learning in the operational phase). We consider that risk culture is driven by four main elements: risk awareness, knowledge, acceptance and memory (Johnson, 1993; Terpstra, 2011).

2. Service Function 2 (SF2): the DICRIM provides information on acting appropriately when facing a major event (decision-making in the operational phase). An appropriate action is based on the way the event is detected and good practices (Bostrom and Löfstedt, 2003).

These are functions fulfilled by the document to satisfy the requirements exerted on it by external circles. Constraints are obtained by examining the external circles in interaction with the document. They are listed in Table 1 (extract).

Table 1 Extract of the table showing requirements features highlighted by the interactions between the DICRIM and external circles.

As shown by Table 1, requirements can be classified into four main groups: regulatory obligations (“compliance with laws”, “in line with the National Model given by MTES”, etc.), content requirements (“provide information on major phenomena”,), form requirements (“to be correctly designed”, etc.), and information flow (“have been received”, “have been kept”, etc.). Three of these groups, regulatory, content and form, will be kept to analyse the requirements at the component level (compliance and internal functional analysis). At this scale, we will not consider the fourth group (information flow) because it does not concern the component level. It is taken into account at a higher level of granularity when the global system is considered.

5 Structural analysis of a DICRIM

Structural analysis aims at listing all the components of the system. The DICRIM components are identified in the National Model issued by the MTES. There are at least 16 components (see Table 2). More components are possible because 6 among those 16 (nos. 5 to 10) are applied for each hazard encountered in the town studied; 10 elements out of 16 remaining do not depend on hazard. For instance, if two hazards threaten the town, there will be 22 components (10 + 2 × 6). That is why there can be as many as 76 (10 + 11 × 6) components in the DICRIM if a town faces all the hazards listed in the National Model (11 hazards; see Sect. 3.1).

Municipalities are free to introduce non-legislated components; however, we do not consider this option at this stage of the analysis.

Table 2 Application of regulatory detection elements in an DICRIM database (extract of eight towns and some detection elements).

Following the requirements described in (31), the DICRIM is composed of an editorial from the mayor, a DICRIM presentation, substantial information for each hazard affecting the town (risk presentation, prevention and protection actions, safety instructions, mapping), information about other preventive informational methods (public notices, floodmarks, etc.), emergency phone numbers and equipment to always have at home to be ready. To facilitate our systemic analysis these requirements are viewed as components. The list of those components is as follows:

• Component 1: cover page (Cp1),

• Component 2: editorial from the mayor (Cp2),

• Component 3: summary (Cp3),

• Component 4: DICRIM presentation (Cp4),

• Component 5: presentation of the risk in the town (Cp5),

• Component 6: prevention actions at town level (Cp6),

• Component 7: police and protection actions (Cp7),

• Component 8: general safety instructions (Cp8),

• Component 9: specific safety instructions (Cp9),

• Component 10: 1 : /25 000 scale mapping (Cp10),

• Component 11: communal poster (Cp11),

• Component 12: floodmarks and highest known flood zone (Cp12),

• Component 13: underground caves and marl pits (Cp13),

• Component 14: where to get more information (Cp14),

• Component 15: emergency phone numbers (Cp15),

• Component 16: equipment to always have at home to be ready (Cp16).

6 Compliance analysis and application to a DICRIM database

To analyse its effectiveness, first we must analyse the DICRIMs compliance with the law by identifying detection elements. This compliance analysis is based on the National Model. Indeed, it was mentioned earlier that government provides some general instructions regarding the content of DICRIM in the form of a National Model. We can think that this model has been strongly discussed in order to make the DICRIM as effective as possible. The regulation detection elements we listed allow the evaluation of the presence or absence of the required content for each component. Components 5 to 10, associated with a given risk, are characterised by 19 detection elements, i.e. 19 elements required by law. With 10 risks being considered here, this gives a total of 190 elements. Component 4 (DICRIM presentation) is characterised by six detection elements. Component 13, specific to underground cavities and marl pits, is broken down into two detection elements. The remaining seven components, with only one detection element each, are not detailed in the National Model. Altogether, there are 205 detection elements to assess compliance with the law for a given DICRIM. Although this number is high, the production of such detection elements simplifies their appropriation if compared to the approach of localising them in the National Model which is very long (360 pages) and where requirements about DICRIM are mixed with DDRM regulations and general information about major hazards. Moreover, as they are presence/absence detection elements, the assessment of the whole sets of symptoms can be quite rapid (∼ 20 min per DICRIM). The completeness of the DICRIM can be verified thanks to them. In our future works they will be formalised as indicators and their number will be greatly reduced, but they can already be used as they are.

To apply those regulatory detection elements, we built a database of 30 DICRIMs. Those DICRIMs were all produced by towns located in the PACA region in France. The DICRIMs were retrieved from municipal websites. A variety of DICRIMs were collected, considering the number and types of hazards affecting the town, the size of the town in terms of number of inhabitants or surface area, the environment of the town (mountain, sea, countryside) and criteria regarding the document itself were also considered. The DICRIMs were more or less long (from 5 to 56 pages – not linked with the number of risks identified). The average length was 20 pages. Their dates of creation also varied with some more being recent than others (from 2003 to 2015).

In this application, we wanted to analyse compliance with regulations for all the DICRIMs. We did not consider mining and volcanic risks because they did not threaten any of the 30 towns studied. That is why 199 detection elements were used during this analysis. Each DICRIM was carefully read and each detection element was assigned 1 if the element was present or 0 if it was absent. The abbreviation “NA” (not applicable) was noted if the hazard asked by certain detection elements was absent in the town studied. The results are presented in table form. An extract from this table is shown in Table 2.

Results highlight that there is a big lack of elements in all the DICRIM reports. In fact, the existing elements seldom reach half of the total requirements for each DICRIM. For instance, town 1 contains 58 of 128 required elements or town 5 only 40 of 125 required elements; ergo in both cases their number is less than half of the elements needed. In conclusion, Table 9 shows that none of the DICRIMs exposed in the database fulfils the elements' requirements.

Elements relative to hazard safety instructions were those which were most present in the DICRIM database. This observation is interesting because safety instructions are major part of one of the two service functions of a DICRIM. The second most common element concerned “the presentation of the DICRIM's role”, an important means of capturing readers' attention.

Table 3 Components and links with service functions.

This analysis also showed that 19 elements required by the National Model were not present in any DICRIM in the database, 22 are only present in one DICRIM and 18 are present in two DICRIMs. These observations lead to questions regarding the relevance of these elements. In particular, one of them was nearly always missing: the role of insurance for each hazard. Some people may think they will be better compensated in case of damage to their property than in reality. It can unconsciously lead them to prepare less for their own safety when a phenomenon is announced. “History of the risk concerned in the municipality by mentioning the most significant events” was also often absent, which was unfortunate in our own opinion. Mentioning significant events is an effective way of raising public awareness of hazards that threaten the town in which they live. It is also a way of preserving memory of the risk. In addition, the lack of a hazard map and a list of equipment necessary in the case of an event is also unfortunate. Indeed, in our opinion they are also important elements for both service functions of the DICRIM.

While these observations show interesting results, we must nevertheless remain cautious about such conclusions with regard to the number of DICRIMs contained in the database. This number is sufficient to test the feasibility of the method, but it will be necessary in future work to increase this number for further results.

The analysis of the presence of elements allows for evaluating to what extent the DICRIM does or does not comply with the regulations. However, this does not necessarily provide any conclusion on the effectiveness of these DICRIMs. The use of other non-regulatory, formal and substantive detection elements is necessary to achieve such a measure. Determination of formal and substantive detection elements needs to identify functions and dysfunctions of components contained in a DICRIM.

7 Component functions: technical function analysis

Technical function analysis (TFA) is the part of functional analysis that helps to formalise and study the architecture of the product (structural analysis) and identify the technical functions of the components (AFNOR, 2015).

7.1 Component/service functions

Regulatory components identified during the structural analysis (see Sect. 5) are listed in Table 3 according to whether they concern the first service function of the system (the global DICRIM) or the second, identified in Sect. 4.

Table 4 List of technical functions identified for both service functions of DICRIM.

Figure 4 Role of the entities of the method.

Some components are specific to one or the other service function. For instance, components 1 to 5 are specific to SF1, while components 15 and 16 are specific to SF2. Indeed, they correspond to emergency phone numbers and equipment, two elements used in case of an event to act appropriately. Some components also perform both service functions. This is the case, for instance, of “policy and protection actions” (Cp7) because they take part in fostering the acceptance of risk, which is one of the elements of risk culture characterising SF1. Indeed, protection actions include the explanation of departmental rescue plans, for example, which make people realise that the risk must be taken seriously. CP7 also performs SF2 insofar as it also gives information on the way the public will be alerted if an adverse event occurs. Alerting is one of the factors that characterise SF2. Finally, all the components are linked to at least one service function.

7.2 Service functions/technical functions

Each component has one or more so-called technical function. They contribute to the service functions. Moreover, they must satisfy the requirements. The technical functions are now linked to the different components: a component fulfils one or more technical functions (Fig. 4).

When establishing the two service functions of the DICRIM, some elements were identified as being linked to both of them. Here, these six identified elements are expressed in terms of six technical functions: four contribute to risk culture and two concern actions during an event. The technical functions of all the components are listed in Table 4 according to whether they concern the first service function or the second.

Table 5 Membership of each technical function identified for the 16 components (TF1: provide information to raise awareness of risk; TF2: provide information to drive knowledge on risk; TF3: provide information to foster the acceptance of risk; TF4: provide information to maintain risk memory; TF5: provide information on event detection; TF6: make known the appropriate responses to adopt if a phenomenon occurs).

Figure 5 Factors influencing a decision-making process related to the occurrence of a phenomenon.

The identification of the technical functions is based on the following elements. The “culture of risk” is not to be seen as a more or less distributed capital but rather as a pragmatic relationship to the danger that is constructed and rebuilt perpetually, sometimes individually, sometimes collectively (CEPRI, 2013). It begins with risk awareness. Knowing of the flood does not imply feeling directly affected by this risk. This awareness of risk has a subjective dimension, specific to each individual or group (leads to define TF1; Defossez, 2012). Conversely, the awareness of what constitutes risk to oneself or to a group cannot be effective without some knowledge of this risk. This knowledge, theoretical and practical and corresponding to TF2, is also built up over time, in particular by means of information received formally or informally, such as the reception of a DICRIM for example. Knowledge and awareness about risk need to be maintained in time, so that it can be forgotten. For a variety of reasons, the transmission of generational knowledge transfer has gradually been extinguished. New populations from urban areas and tourists are often unaware of the risk their municipality is exposed to. Moreover, a phenomenon may possess a very long return period, even leading long-settled populations to forget the danger still present CEPRI (2013). This leads to TF4, characterising memory of risk. To expect an appropriated behaviour, acceptance of risk is also crucial (TF3). Again, this process is strongly related to the nature of the risk communication that has been carried out, including the credibility of the source. When an individual feels vulnerable to risk, deprived and without means to cope with risk, they may have more difficulty accepting the risk and tend to take refuge in the denial of that risk. Denial is part of a set of perception biases (overconfidence, anchoring effect, etc.), which can occur in the face of risk, running counter to the ingredients described above and which build the culture of risk (awareness, knowledge, memory, acceptance; Fig. 5; Mary and Wildavsky, 1987).

In many cases, cognitive bias takes precedence over the rationality of a decision (Slovic, 1987). For example, over-confidence corresponds to an overestimation of ones knowledge or an underestimation of ones uncertainty (Kahneman and Tversky, 1979). Most of the time subjects underestimate what they do not know. Preventive risk information systems such as the DICRIM therefore exist with a view to avoiding, as much as possible, bias taking precedence in a decision-making context. The DICRIM would therefore contribute to the risk culture by deepening these major ingredients by using technical functions. The DICRIM also provides information on acting appropriately when facing a major event. An appropriate action is based on the way the event is detected (leading to TF5) and good practices characterising TF6 (Bostrom and Löfstedt, 2003).

Table 6 An extract of some content requirements satisfied by each component.

Table 5 represents the membership of each technical function identified for the 16 DICRIM components. We consider that components 5 to 10 generally concern each hazard. Generally, all the functions are covered by at least five components and all the components perform at least one function. TF1 is filled with 10 components. This is the function performed by most of the components. Cp14 satisfies all the technical functions of both service functions. Components 8, 9, 15 and 16 perform only one function (TF6). The pairs of components 1/2, 3/4 and 12/13 fulfil the same functions. Cp5 performs all the technical functions linked to SF1 and Cp11 the two technical functions of SF2.

The redundancy in this table is obvious. However, hasty conclusions must not be drawn from these observations. For example, although two components fulfil a single function, this does not mean that one duplicates another. Depending on the context, accomplishing the same function through several components may be beneficial. For example, awareness may not be immediate and require the addition of more information. Conversely, too much knowledge can cause boredom for the reader who may lose interest in the topic or the whole component. A detailed analysis will be performed when identifying adjustments to the content of the regulatory document. This concerns a work in progress, not presented here, that consists of interviewing the population about their perception of the DICRIM.

Table 7 Functional analysis table for Component 2.

Table 8 Example of an indicator grid for evaluating the length of the editorial component.

Figure 6 Diagram with a bow tie showing FMEA for Component 2.

7.3 Constraints satisfied by the components

To fulfil its technical function, a component must conform to one or more requirements (see Fig. 4). The respect for the requirements by components are evaluated by detection elements that will be showed later in the article. The first requirements are regulatory ones but during our analysis we found that these requirements were not sufficient when the aim is to evaluate the effectiveness of the DICRIM. Other requirements were identified in the EFA when listing the requirements exerted on the DICRIM by external circles. The same groups must be considered here regarding the scale of the components. They are content requirements and form requirements identified in the literature (communication, advertising, etc.). It was necessary to add form requirements because the National Model does not include any specific instruction on this part, though it plays a crucial role in the effectiveness of a communication. It was also necessary to add content requirements because it remains essential to include details for effectiveness not covered by the regulatory requirements. Table 6 shows an extract of requirements filled by each component relative to content characteristics. Table 6 is composed of a total of nine columns. Each component satisfies at least one constraint. The first three DICRIM components must persuade the reader to continue reading the DICRIM because they are usually the first items they see when they open the document. There are, however, no regulations on these three components. The designer is therefore free to develop the DICRIM. Each component must capture attention to be read. The same type of table was produced for form requirements (six columns).

7.4 Functional analysis table

For each component, the technical functions along with the service function they refer to and the requirements are then gathered in functional analysis tables. Table 7, showing the component “editorial with a word from the mayor”, is an example.

Figure 7 Example of the functioning of the future effectiveness model by indicator aggregation.

8 Causes, effects and detection of components failures

The results produced by the functional analysis form the basis for performing an FMEA. The FMEA is carried out for each function performed by the components of the document. All the failure modes (that is to say functions not carried out) that can occur during the different life cycles of the document and their causes and effects are identified. It is essential to investigate the causes and effects of dysfunctions carefully to detect them and then propose feedback to avoid them. Effects are not used in our work but they will be important results in the future development of the effectiveness evaluation model. Furthermore, as we said before, a component must satisfy a criterium to perform a technical function. Non-compliance with the criteria is seen here as a cause of dysfunction (see Fig. 4) and they are therefore essential to identify detection elements. Compliance with the requirements is evaluated by one or more “detection elements” (Fig. 4).

The results of the FMEA are presented in a diagram with a bow tie form. Figure 6 presents an example for the editorial component.

An editorial with a word from the mayor is considered a failure when its functions “provide information to raise awareness of risk” and “provide information to foster the acceptance of risk” are faulty. It occurs when form and content of the editorial are not complied with requirements. To detect those failures form detection elements and content detection elements are used to evaluate the typography, the text length, the vocabulary, etc. Those failures can have some negative effects. For instance, as the editorial is not effective, the reader does not view this section and might not want to continue to read the DICRIM or even not accept the risks present on its town.

To detect failures of components we identify detection elements regarding form and content. The scales of indicators are based on literature and experts. These scales require the mobilisation of different disciplines such as semiotics with for example the improvement of colours (Chesneau, 2006), or the study of the zones of interest of an image. Judd et al. (2012) study, for example, the ability of visual salience models to predict areas of interest in an image. Fabrikant et al. (2010) study how visual salience and training on cartographic reading impact the effectiveness of a map. Using an experimental protocol studying eye movements, understanding and answering volunteers' questions on different types of maps, Palka (2015), for example, showed that the most useful maps are characterised by a quantity of information contained in them. The location of the fixations also allowed the author to define which areas of the map attract the eye with their graphic or informational characteristics. This knowledge from different fields will in fact be exploited in more detail in our future work during the development of the indicators.

For form, 13 generic detection elements were identified. They are used for one or some of the 16 DICRIM components according to what they allow one to determine. The same approach is followed for the seven content detection elements identified. In future works, content and form detection elements will be formalised as indicators and presented in a grid form. An example of those indicators is shown in Table 8. It allows for evaluating the length of the component, here the editorial component, regarding number of pages combined with the font size. This is one of the important features to take into account for evaluating its effectiveness.

Then all those form and content indicators will be aggregated to form an effectiveness detection model. Several aggregation operators can be used such as weighted average, minimal, etc. In Fig. 7, an example of the effectiveness model is showed. It contains indicators with their hypothetical weights, used to demonstrate how a “bad” score becomes useful for detecting where feedback must be applied to improve a given component's overall score.

9 Conclusion and outlook

Providing information and communication is essential for raising awareness of risks and disseminating knowledge on their nature and on how to act if a phenomenon occurs. That is why it is crucial to ensure that relevant information and communication is available to those who would use them. It allows decisions to be taken without bias. However, few works currently propose to evaluate their effectiveness. Existing works often require a sample population or are expensive to perform. According to the means available, such studies are not always possible. Furthermore, these methods are specific to the sites where they must be applied and are not necessarily generic. In this study we proposed an analytical methodology that allows identifying preventive communication documents functions and dysfunctions. It lies at the interface between several existing approaches, from different fields (engineering, advertising, etc.). This method was applied to the DICRIM, one of the main French documents used to inform the general public about risk at a communal level. Results included the identification of regulatory detection elements. These detection elements were then used to analyse existing documents and show their degree of conformity with the regulations. A challenge in the future will be to analyse whether component redundancy is strictly necessary (Table 5). Indeed, we observed that some components fulfilled the same function, raising doubts as to their usefulness. Likewise, the application of the database showed that several elements demanded by the regulations were systematically missing from the DICRIMs studied. This also led to questioning the need for these elements to fulfil the objectives of the document.

The approach is generic and could be applied to other documents, notably for preventive risk management such as the family safety plan intended to help families. Families have to complete it themselves in order to prepare for the possible occurrence of an event. Our approach and its general procedure (EFA and FMEA), service functions, types of requirements, form requirements and some of regulatory and content requirements may be applied to the case of family safety plans.

This study is the first step towards a decision support model for the municipality (or specialised consulting firms) in charge of drawing up documents. This model will allow for the evaluation of the effectiveness of existing preventive communication documents and identification of the corrective actions needed to improve their effectiveness. In the next step (not presented here), the causes and effects identified with the FMEA method will be used to define models for the quantitative assessment of DICRIM efficiency. Detection elements will become formalised indicators, which will be used as input data in these models. This is an essential step towards the overall goal. Its results are crucial for ensuring the basis of models and for structuring them.

Appendix A: Table of abbreviations in the article

Abbreviation	Meaning
DICRIM	Document d'Information Communal sur les Risques Majeurs
	(Municipal Information Document on Major Risks)
DDRM	Document départemental sur les risques majeurs
	(Departmental Document on Major Risks)
EFA	external functional analysis
FMEA	failure modes and effects analysis
IAL	Information Acquéreur Locataire
	(Purchaser Tenant Information)
MTES	Ministère de la Transition Ecologique et Solidaire
	(Ministry of Ecological and Inclusive Transition)
OAP	Orientations d'Aménagement et de Programmation
	(Development and Planning Guidelines)
PAC	Porté à Connaissance
	(Knowledge of existing data)
PFMS	Plan Famillial de Mise en Sûreté
	(Family Plan for Safety Layout)
PPRI	Plan de prévention des risques inondations
	(Flood Risk Prevention Plan)
SCoT	Schéma de Cohérence Territoriale
	(Territorial Coherence Scheme)
SF1	Service Function 1
SF2	Service Function 2
TFA	technical function analysis