Framework to prioritize watersheds for diffuse pollution management in the Republic of Korea: application of multi-criteria analysis using the Delphi method

This study aimed to develop a risk-based approach for determining control areas to manage non-point source pollution, developing a framework to prioritize catchments by considering the characteristics of polluted runoff from non-point sources. The best management, decision-making, and scientific approaches, such as the technique for order of preference by similarity to ideal solution (TOPSIS) and the Delphi technique, are required for the designation of control areas and the application of the best management practices to the control areas. Multi-criteria decision-making (MCDM) methods can handle the diversity and complexity of non-point source pollution. The Delphi technique was employed for selecting the assessment criteria/sub-criteria and determining their weights. Sub-criteria for each catchment unit were scored with either a quantitative or qualitative scale. All non-point pollution sources in mainland Republic of Korea were included, with the exception of a few islands, with catchment prioritization and pollution vulnerability evaluations shown as thematic maps. This study contributes to the field by developing a new risk-based approach for ranking and prioritizing catchments; this provides valuable information for the Ministry of Environment to use to identify control areas and manage non-point source pollution.

1 Introduction

Diffuse pollution (non-point source pollution) is a major issue in water quality management and catchment management (Hoppe et al., 2004; Huang and Xia, 2001; Lee and Bae, 2002; Orr et al., 2007). Pollutants accumulated from diverse diffuse sources generally move by runoff and make water quality problems worse. Water quality problems caused from diffuse pollution are influenced by meteorological, hydrologic, and demographic characteristics of catchments. Insights and tools addressing complexity and uncertainty of the problems are required to solve the problems.

The diversity and complexity of diffuse pollution can be described by catchment-based risk assessments (Candela et al., 2009; Wang and Yang, 2008) and the assessments can be undertaken by multi-criteria analysis (Huang et al., 2013). The multi-criteria analysis is suitable to develop consultation for management on a complicated system. The approach is practical to deal with many decision problems in environmental management which involve multiple conflicting evaluation criteria as well as a large number of spatial units (Zhang and Huang, 2011). Diverse methods were applied broadly in water-related matter because catchment-based water management is complex and interactive due to the inherent trade-offs between social, political, ecological, and economic factors (Kiker et al., 2005). Giupponi and Rosato (1999) developed a multi-criteria analysis system for producing risk maps of agricultural pollution. Munafo et al. (2005) developed a potential non-point pollution index (PNPI) to assess the global pressure on surface water bodies. Zhang and Huang (2011) developed a GIS-based multi-criteria analysis method to assess the potential contributions of different land areas in diffuse nutrient export at the basin scale. Chun et al. (2012) took a risk-based approach to prioritize catchments for diffuse metal pollution management. Huang et al. (2013) employed multi-angle indicators of non-point source pollution, deficient waste treatment, and public awareness of environmental risk to identify key environmental risk sources contributing to water eutrophication and to suggest certain risk management strategies for rural areas.

The Ministry of Environment (MOE) of the Republic of Korea has tried to deal with diffuse pollution issue after the 2000s. The MOE currently enforces programs for the control of diffuse pollution under the Water Quality and Ecosystem Conservation Act (WQECA): the reporting on facility installation to reduce diffuse pollutants from new development sites or industrial sites, the designation and management of control areas to be required to manage diffuse pollution, and so on. The programs have been implemented under insufficient data, tools, information, and knowledge for diffuse pollution. It makes it difficult to assess existing diffuse pollution, to establish the measures including selection and spatial allocation of management practices, or to evaluate the measures. Recently, the lack of a sound decision-making system and efficient resource allocation have been significant issues in South Korean diffuse pollution management. The current decision-making support system should be reorganized based on expert advice and a scientific basis for more efficient policy implementation.

We were interested in the development of a decision-aiding tool for the selection/designation of control areas. This study aimed to develop a framework to evaluate and to prioritize South Korean watersheds in terms of the need for diffuse source management. Moreover, the Delphi method to obtain the most reliable consensus of a group of experts (Dalkey and Helmer-Hirschberg, 1962; Linstone and Turoff, 1975; Okoli and Pawlowski, 2004) was employed to reflect experts' opinions in the framework development.

2 Methodology

2.1 Background and study procedure

The Ministry of Environment in the Republic of Korea has a plan to continuously expand diffuse pollution management areas until 2020. In 2009, the candidate management areas for industrial sites and small watersheds had been additionally determined for conducting a feasibility study and preparing selection criteria. Relevant authorities also established the Second Comprehensive Plan for Diffuse Pollution Management (from 2012 to 2020) in a collaborative project. As a part of this plan, an (draft) improvement scheme for the criteria of determining and assessing management areas has been prepared and implemented in order to expand diffuse pollution management areas and improve the related systems. Accordingly, the order of priority needs to be set on a scientific basis so that areas which require diffuse pollution management most urgently will receive be systematic preference. For diffuse pollution policy, a vulnerability analysis and a map of vulnerable areas are required.

Some studies (Bang et al., 1997; Yoon et al., 2007; Choi et al., 2009; Park et al., 2014) by the Ministry of Environment and the Korea Environment Corporation have been conducted so far on the nomination of diffuse pollution management areas. However, most of these studies assumed scenarios after considering significant factors and merely analyzed the scenarios to predict the results of assumptions. Consequently, determining, allocating, and scoring the evaluation items and weights have become key research issues for the identification of areas vulnerable to non-point source pollution. To identify and prioritize vulnerable areas for diffuse pollution management, the items to be evaluated as well as the weighting and scoring methods for each item should be importantly considered and determined. In this regard, many experts have constantly expressed the opinion that research and surveys are needed to propose specific indexes quantifying the contribution rates and weights, and to determine a selection method for vulnerability assessment (Park et al., 2013).

The degradation of water quality and aquatic ecosystem due to diffuse pollution sources is related to uncertain factors such as emission characteristics of various pollutants, and climate and soil properties. Naturally, diverse solutions are being proposed by experts and interested parties, which prevents clear policies from being firmly implemented. For efficient policy implementation, a quantitative, objective, and scientific analysis needs to be conducted for factors causing diffuse pollution, and then the management areas should be expanded. This will result in not only systematic policy implementation but also highly efficient, low-cost investment.

The process of the study to prioritize watersheds for diffuse pollution management in South Korea is presented in Fig. 1.

• Step 1 sets an evaluation framework and performs a Delphi survey for experts to determine evaluation items and weights.

• Step 2 collects and quantifies data of each evaluation item for each watershed. The technique for order of preference by similarity to ideal solution (TOPSIS), which is one of multi-criteria decision-making (MCDM) techniques, is applied for evaluation.

• Step 3 prepares the vulnerable area map by using evaluation results, and selects major vulnerable areas.

Figure 1 Study procedure.

2.2 Determining criteria with modified Delphi method

The Delphi method was developed by the RAND Corporation in the 1950s aiming to reduce the range of group responses and to strive for expert consensus. The Delphi method as a method for structuring a group communication process is accomplished by feedback of individual contributions of information and knowledge, assessment of the group judgment or view, an opportunity for individuals to revise views, and a degree of anonymity in individual responses (Linstone and Turoff, 1975). A series of questionnaires with controlled opinion feedback is typically used for collecting and distilling knowledge from a group of experts (Rowe et al., 1991; Adler and Ziglio, 1996; Angus et al., 1996). The process by which experts reply to questionnaires, subsequently receive feedback, and modify their opinion is repeated until arriving at the most reliable consensus.

The Delphi method is effective in allowing a group of individuals, as a whole, to deal with a complex problem (Mohorjy and Aburizaiza, 1997) and has been applied in various fields such as information systems, planning, environmental impact assessments, social policy, and public health (Angus et al., 1996; Linstone and Turoff, 1975; Okoli and Pawlowski, 2004). Also, there have been several applications in studies on water resource use and management, water quality assessment, and so on (Cude, 2001; Kim and Chung, 2013; Lee et al., 2013; Parparov et al., 2006; Parparov and Hambright, 2007).

For successful progress, there are two important considerations. The first is to select experts participating in our survey. We considered that the respondents should be experts with plenty of experience and with a high level of responsibility. The next important consideration is to reach an agreement involving experts. Obtaining consensus will take a long time because every expert has their own opinion, and these are sometimes extremely different.

We have simplified the Delphi process. The nub of modified Delphi procedure is that experts were provided detailed and concrete information for candidate criterion by organizing group. Figure 2 explains the procedure of our modified Delphi.

Figure 2 Delphi survey procedure.

Firstly, we selected a group of experts in diffuse pollution management of South Korea. The experts are ones who have extensive experiences with a high level of responsibility or have carried out a lot of research on diffuse pollution. Then we created an evaluation framework with the candidates of criteria and sub-criteria after examining literature and brainstorming. This is because it is time-consuming and inefficient to reach an agreement after every expert suggests an evaluation framework. A questionnaire was prepared to obtain expert opinions for the framework's structure, and the candidate criteria and sub-criteria. After collecting and analyzing the judgments of a group of experts, the evaluation framework consisting of criteria and their weights was determined if the consensus of the group emerged.

2.3 Assessing potential risk with TOPSIS

In this study, the criteria scores were estimated by the TOPSIS method of multi-criteria decision analysis methods (Fishburn, 1967; Hwang and Yoon, 1981). The TOPSIS chooses the alternative of the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution (Lai et al., 1994; Chu, 2002; Jun et al., 2011; Lee et al., 2013).

In addition, assessment results for all alternatives can be easily calculated and presented from a multi-attribute perspective (Kim et al., 1997; Shih et al., 2007; Lee and Chung, 2007; Chung and Lee, 2009).

Here the positive ideal solution (PIS) is the most vulnerable area and the negative ideal solution (NIS) is the least vulnerable area. The TOPSIS procedure is as follows:

Construct the weighted decisions matrix (v_ij):

$$\begin{array}{}\text{(1)}& {\mathbf{v}}_{ij}=w_i\times x_{ij},\end{array}$$

where w_i is the weight of ith criterion, x_ij are built by alternative $A_j(j=\mathrm{1},\mathrm{\dots },n)$, which are evaluated against criteria $C_i(i=\mathrm{1},\mathrm{\dots },m)$, the standardized data of each assessment unit area.

Determine the PIS(A⁺) and NIS(A⁻) of unit area:

$$\begin{array}{}\text{(2)}& {\displaystyle }& {\displaystyle }{A}^{+}=v_{\mathrm{1}}^{+},\mathrm{\dots },v_n^{+},\text{(3)}& {\displaystyle }& {\displaystyle }{A}^{-}=v_{\mathrm{1}}^{-},\mathrm{\dots },v_n^{-}.\end{array}$$

Here $v_i^{+}=\max \left({\mathbf{v}}_{ij}\right)$, $v_i^{-}=\min \left({\mathbf{v}}_{ij}\right)$

Calculate the distance from the positive ideal ($d_{\mathrm{i}}^{+}$) and the negative ideal ($d_{\mathrm{i}}^{-}$) solution for each alternative

$$\begin{array}{}\text{(4)}& {\displaystyle }& {\displaystyle }{d}_{\mathrm{i}}^{+}={\left\{\sum _{j=\mathrm{1}}^n{\left({\mathbf{v}}_{ij}-v_j^{+}\right)}^{\mathrm{2}}\right\}}^{\mathrm{1}/\mathrm{2}},i=\mathrm{1}\mathrm{\dots },n\text{(5)}& {\displaystyle }& {\displaystyle }{d}_{\mathrm{i}}^{-}={\left\{\sum _{j=\mathrm{1}}^n{\left({\mathbf{v}}_{ij}-v_j^{-}\right)}^{\mathrm{2}}\right\}}^{\mathrm{1}/\mathrm{2}},i=\mathrm{1}\mathrm{\dots },n.\end{array}$$

Calculate the optimum membership degree ($D_i^{+}$):

$$\begin{array}{}\text{(6)}& D^{+}={\displaystyle \frac{d_{\mathrm{i}}^{-}}{d_{\mathrm{i}}^{+}-d_{\mathrm{i}}^{-}}},(i=\mathrm{1},\mathrm{\dots },m).\end{array}$$

The priority of watersheds in terms of the need for diffuse pollution management was decided according to the criteria scores aggregated for watersheds. The priority of diffuse pollution management was represented in a map with a geographic information system (GIS).

3 Application

3.1 Study area

The MOE shall regularly survey the kinds of sources of pollution in order to ascertain the current status of water quality and aquatic ecosystems by river-system spheres of influence and shall develop the basic plan for preserving water quality and aquatic ecosystems (WQECA, 1997, Article 22 and Article 23). The spatial extent of the investigation is the whole country of South Korea.

The river-system spheres of influence are classified as small, medium, and large areas of influence. In the study, the potential risk of diffuse pollution was evaluated for 814 watersheds which are the small areas of influence and the subjects of the pollution survey.

3.2 Determination of the evaluation framework by the Delphi survey

The evaluation framework, criteria, and their weights were determined by the experts' agreement through the Delphi survey. The weights of multi-criteria were decided by the ranking method.

A nonpartisan expert pool was asked whether they would participate or not. A total of 12 experts gave us a positive answer. They worked in the following different sectors: government-funded research institutes (8 %), private engineering companies (17 %), public services (17 %), and university (58 %). All had doctoral-level training and most (80 %) have been researching diffuse source pollution management for over 10 years.

Candidates of criteria and sub-criteria were organized based on brainstorming with literature research. Then, the draft questionnaire including candidate criteria was distributed for experts. This was the start of the first round. After we collected and analyzed the raw data from the questionnaires, we then revised the questionnaire. The modified questionnaire included the analysis result of previous survey. The round proceeds in the same way, continued until consensus emerges.

A draft framework to prioritize the watersheds by evaluating the potential risk of diffuse pollution was developed in consideration of the availability of data related to diffuse pollution in South Korea and the characteristics of diffuse pollution discussed in other studies (Chun et al., 2012; Novotny, 2002; Jang et al., 2012; Jung et al., 2011; Park et al. 2010). Although the diffuse pollution is irregular, variable, and indefinable and its risk varies depending on the watershed (Candela et al., 2009; EA, 2007; US EPA, 1997), the “source–pathway–receptor” concept is applicable and useful for the evaluation. Chun et al. (2012) defined criteria of activities and land-use representing pollution sources, rainfall, and runoff characteristics, and physical, chemical, and ecological status of the receptor adopting the concept. Similarly, Jang et al. (2012) used the characteristics related to the process of generation, discharge, and delivery to the receiving waters of agricultural areas. In this study, the pollution sources, hydrologic processes, and status of receiving water were employed as the groups used to classify the criteria.

The criteria of land use, activities in urban areas and agricultural areas, and the sub-criteria such as population density, livestock numbers, fertilizer use, and area of different land use were selected for the pollution source; the criteria of rainfall and runoff and sub-criteria such as annual rainfall, rainy days, drainage area, and runoff ratio for the hydrologic processes; the criteria of water resources, water quality, and aquatic ecosystems and the sub-criteria such as river flow, water quality based on biochemical oxygen demand (BOD), suspended solids (SS), and the indicator of aquatic ecosystem health were selected for the status of the receiving water. The criteria and the sub-criteria of the draft of evaluation framework are as shown in Table 1.

Table 1 A set of criteria and sub-criteria for the draft of evaluation framework.

The Delphi survey was carried out to get the approval of experts for the evaluation framework. We selected a total of 13 experts who showed interest in participating in the survey. The experts, with experience, education, and training at the doctoral-level, work for government-funded research institutes, government-affiliated organizations, or universities, and most of them have been involved in diffuse source pollution management for over 10 years. The experts were asked to check the structure of the evaluation framework, to exclude/add the criteria and the sub-criteria, and to decide their weight. At the first round, they supported the group and the criteria of the draft of evaluation framework but requested to modify some of the sub-criteria. The sub-criteria were classified with “Acceptance”, “Rejection”, and “Addition” and were revised for the second round. The feedback on the modified questionnaires was positive and the experts' consensus was built as shown in Table 2.

Table 2 Determination of the evaluation framework by the Delphi survey.

In this study, two types of the weight sets selected by the expert panels are used. The first type is the ranking sets ($w_i^{\mathrm{rank}}$). For a given set of variables, each participant determines the ranking in the order of its highest value for each of the factors in a manner that determines and quantifies its importance. Thus, the most important factor becomes rank 1, and the next most important one is rank 2. At this point, the component may be ranked in the same order. The constructed ranks are aggregated through the conversion, with Rank 1 being converted into m−1 and Rank 2 being converted into m−2, where m is the total number of factors. Calculate the order of these conversions by Eqs. (5) and (6).

$$\begin{array}{}\text{(7)}& {\displaystyle }& {\displaystyle }{R}_i=\sum _{j=\mathrm{1}}^n{R}_{ij},\text{(8)}& {\displaystyle }& {\displaystyle }{w}_i^{\mathrm{rank}}=R_i/\sum _{i=\mathrm{1}}^m{R}_i,\end{array}$$

where R_i is the sum of transformed ranks, R_ij is the rank that was selected by the jth panel of experts, and m is the total number of criteria, n is the total number of expert panels.

The weights of the group, the criteria, and the sub-criteria were determined by the ranks suggested by the experts. The experts judged that the pollution source (0.4853) is more important than the hydrologic process or the receiving water in the group.

The second type of weights set is the rating sets ($w_i^{\mathrm{rate}}$). It is a way to compare importance of criteria to distribute weights. The survey respondents will determine the weights for each criteria, but will then select values within the range given. The range of weights is a continuous section, generally ranging from 0.0 to 1.0 or up to 100.0. In addition, the sum of the weights given to all variables under comparison is equal to the maximum range given. A factor equivalent to 0.0, the lowest limit of the section, is of no importance to the assessment, whereas a maximum value means that a maximum number of possible values of significance are applied.

The weights may be calculated from Eqs. (7) and (8). It is also possible to set the same value by allocating the relative importance of each factor.

$$\begin{array}{}\text{(9)}& {\displaystyle }& {\displaystyle }{w}_{ij}=p_{ij}/\sum _{i=\mathrm{1}}^m{p}_{ij},\text{(10)}& {\displaystyle }& {\displaystyle }{w}_i^{\mathrm{rate}}=\sum _{j=\mathrm{1}}^n{w}_{ij}/\sum _{j=\mathrm{1}}^n\sum _{i=\mathrm{1}}^m{w}_{ij},\end{array}$$

where p_ij is the weight of criteria i, as determined by panel j. Established weights are shown in Table 3.

Table 3 Determination of the weights by the Delphi survey.

The evaluation framework thus established can be applied flexibly in various conditions including securing of relevant data. In other words, if data are insufficient or uncertain, evaluations are conducted either by removing or applying such insufficiency or uncertainty, and the evaluation results are analyzed to improve the framework. This “adaptive management” method is an iterative approach (Holling, 1978; Walters, 1986) that enhances management ability by accumulating an accurate understanding and knowledge of response for a target system.

3.3 Collection, quantification, and standardization of evaluation data

3.3.1 Data collection and quantification for each evaluation item

Data such as population density, urbanization level, and fertilizer use were collected from the source of either the statistics of local governments or Statistics Korea (Republic of Korea), and the precipitation data including annual rainfall were provided by the Korea Meteorological Administration. In addition, a land use map, industrial conditions, combined/sanitary sewer systems, livestock numbers, livestock barn area, a watershed map, water resources, and water quality were derived from the data surveyed by the Ministry of Environment (Table 4).

When values of each evaluation item for 814 small watersheds were determined, some data items were not measured or missed. As for the water quality data, if there was a water quality observatory in a watershed, the data were obtained from it, and if there was no such observatory and thus no measurement was available, data from an adjacent watershed or lake were analyzed and utilized. As for flow rate, if a small watershed consists of a single basin, the measurement of flow rate is attributable to the watershed. On the other hand, if there was another upstream small watershed, the data measured in a downstream small watershed cannot represent the characteristics of all small watersheds. In order to respond to this problem, the flow rate, rainfall, and areas of the upstream basin and small watershed were used to calculate a specific discharge and determine the flow rate of each small watershed.

$$\begin{array}{}\text{(11)}& Q_{\mathrm{2}}={\displaystyle \frac{P_{\mathrm{2}}}{P_{\mathrm{1}}}}{\displaystyle \frac{A_{\mathrm{2}}}{A_{\mathrm{1}}}}{Q}_{\mathrm{1}},\end{array}$$

where Q is flow rate, P is rainfall, and A is the basin area at the calculation point of flow rate. The subscript 1 means the reference point, and subscript 2 indicates the calculation point of flow rate.

Since many small watersheds including estuaries do not have any measurements, even at the middle watershed level, such watersheds were left unmeasured and a low score was given without using the data of adjacent small watersheds. In addition, if necessary, the flow rate data of a dam were also utilized to represent runoff characteristics of small watersheds.

Collected data were spatially distributed as Fig. 3.

Figure 3 Sample of collected data sets.

3.3.2 Standardization of evaluation items

Because each data set for evaluation items has different units and properties, standardization is required to use data sets for evaluation. The re-scaling method was adopted in the standardization process. The overall range of data was normalized to assign values between 0 and 1, as described in Eq. (10).

$$\begin{array}{}\text{(12)}& X_i={\displaystyle \frac{x_i-x_{\min }}{x_{\max }-x_{\min }}},\end{array}$$

where X_i is the ith standardized value, x_i is the ith data value, x_max is the maximum value, and x_min is the minimum value.

However, in case the data collected are used to standardize evaluation items without modification, the standardized scores are often either biased or equalized in their range and distribution according to characteristics and types of data.

Accordingly, since it seemed to be unreasonable to apply the above equation without modification, the data collected were prioritized and the consequential order of priority was scored before the equation was used for standardization. The detailed modification process is shown in Table 4.

3.4 Assessing vulnerability

The vulnerability of every small watershed to diffuse pollution was evaluated by using data and weights for each factor, and the vulnerable areas were determined based on this assessment (Fig. 4). In addition, the small watersheds were prioritized again as part of each of four large watersheds, and top (more vulnerable) 30 small watersheds (of these large watersheds) are illustrated in Fig. 5. This was because the pollution source management and relevant policies were organized based on the large watersheds. Both ranking and ratio methods were applied to calculate weights.

Figure 4 Spatial diffuse pollution vulnerability results for South Korea.

Figure 5 Top 30 small watersheds of diffuse pollution vulnerability for each of the four large watersheds.

Among top 50 small watersheds in the order of priority in each large watershed, main rivers and small watersheds, which required diffuse pollution source management, were derived in each river system.

The Han River basin has three priority control target rivers: downstream of Namhan River, middle of the Han River, and Anseong Stream. The Geum River basin has four priority control target rivers: midstream of the Geum River, Dongjin River, Mankyeng River, and Sapgyo Stream.

Youngsan River basin has two priority control target rivers: Youngsan River and Sumjin River. The most vulnerable area of the Nakdong River basin is in main stream of Nakdong River.

The evaluation results were analyzed in terms of effects of each evaluation factor. It turned out that if a large number of livestock are reared and much fertilizer is used in a basin, the land area is wide, and the public water has much soil and a high SS concentration, such a watershed needs to be preferentially managed.

Table 4 Methods of collection and quantification for the criteria.

4 Conclusion

There are few studies to assess watersheds in respect of diffuse pollution management in South Korea. This study has suggested a scientific analysis method for selecting priority areas in the current diffuse pollution management system. As various uncertain factors are included in assessing vulnerable areas to diffuse pollution sources, such factors need to be quantified and analyzed objectively and scientifically. The Delphi method was used to determine the vulnerability of the evaluation items, which included basin characteristics, pollution source, and water quality, and weights for diffuse pollution, on the basis of expert opinions. Criteria and sub-criteria were allocated into three groups of pollution source, hydrologic process, and receiving water. Based on the weights and evaluation items thus obtained, data from each item were applied, and the vulnerability to diffuse pollution was assessed by the TOPSIS method. The proposed evaluation process will promote efficient policy implementation and set a foundation for scientific/clear diffuse pollution management.

In addition, this study attempted a small watershed-based analysis for more selective/intensive policy enforcement. However, it was difficult to standardize quantitatively each evaluation item, in the determination of management areas, at the level of small watersheds.

Accordingly, a runoff model needs to be applied to improve the estimations for unmeasured areas. A vulnerability assessment system for diffuse pollution is also to be established in order to promote efficient policy enforcement. Such a system should update relevant data and enable cyclic reevaluation.

Finally, this study has not reflected the current diffuse pollution management policy in the list of evaluation items. This was because the effect of the policy could not be accurately quantified. A further study will solve this problem and include the current policy in its assessment.