Framework to prioritize watersheds for diffuse pollution management in Korea: application of multicriteria analysis using the Delphi method

10 This study aimed to develop a risk-based approach for determining control areas to manage non-point source pollution, developing a framework to prioritize catchments by considering the characteristics of polluted runoff from non-point sources. The best management, decision-making, and scientific approaches, such as TOPSIS and Delphi technique, are required for the designation of control areas and the application of the best management practices to the control areas. Multicriteria decisionmaking methods can handle the diversity and complexity of non-point source pollution. The Delphi technique was employed 15 for selecting the assessment criteria/sub-criteria and determining their weights. Sub-criteria for each catchment unit were scored with either a quantitative or qualitative scale. All non-point pollution sources in South Korea mainland were included, with the exception of a few islands, with catchment prioritization and pollution vulnerability evaluations shown as thematic maps. This study contributes to the field by developing a new risk-based approach for ranking and prioritizing catchments; this provides valuable information for the Ministry of Environment to use to identify control areas and manage non-point 20 source pollution.


Introduction
Diffuse pollution (Non-point source pollution) is a major issue in water quality management and catchment management (Hoppe et al., 2004;Huang and Xia, 2001;Lee and Bae, 2002;Orr et al., 2007). Pollutants accumulated on diverse diffuse sources generally moves by runoff and makes water quality problems worse. The water quality problems caused from diffuse 25 pollution are influenced by meteorological, hydrologic and demographic characteristics of catchments. Insights and tools addressing complexity and uncertainty of the problems are required to solve the problems.
The diversity and complexity of diffuse pollution can be described by Catchment-based risk assessments (Candela et al. 2009; Wang and Yang, 2008) and the assessments can be undertaken by multi-criteria analysis (Huang et al., 2013). The multicriteria analysis is suitable to draw a consultation for management on a complicated system. The approach is practical to deal with many decision problems in environmental management which involve multiple conflicting evaluation criteria as well as a large number of spatial units (Zhang and Huang, 2011). Diverse methods were applied broadly in water-related matter because catchment-based water management is complex and interactive due to the inherent trade-offs between social, political, ecological and economic factors (Kiker et al., 2005). Giupponi et al. (1999) developed a multi-criteria analysis system for producing risk maps of agricultural pollution. Munafo et al. (2005) developed a potential non-point pollution index (PNPI) to 5 assess the global pressure on surface water bodies. Zhang and Huang (2011) developed a GIS-based multi-criteria analysis method to assess the potential contributions of different land areas in diffuse nutrient export at the basin scale. Chun et al.
(2012) took a risk-based approach to prioritize catchments for diffuse metal pollution management. Huang et al. (2013) employed multi-angle indicators of non-point source pollution, deficient waste treatment, and public awareness of environmental risk to identify key environmental risk sources contributing to water eutrophication and to suggest certain risk 10 management strategies for rural area. the report on facility installation to reduce diffuse pollutants from new development sites or industrial sites, the designation and management of control areas to be required to manage diffuse pollution, and so on. The programs have been implemented 15 under insufficient data, tools, information and knowledge for diffuse pollution. It makes it difficult to assess existing diffuse pollution, to establish the measures including selection and spatial allocation of management practices, or to evaluate the measures. Recently, sound decision making system and efficient resource allocation are significant issues in the Korean diffuse pollution management. The current decision making support system should be reorganized based on expert advice and scientific basis for more efficient policy implement. 20 We were interested in the development of a decision-aiding tool for the selection/designation of control areas. This study aimed to develop a framework to evaluate and to prioritize Korean watersheds in terms of the need of diffuse source management.
Moreover, the Delphi method to obtain the most reliable consensus of a group of experts (Dalkey and Helmer, 1963;Linstone andTuroff, 1975, 2002;Okoli and Pawlowski, 2004) was employed to reflect expert's opinions in the framework development.

Background and study procedure
The Ministry of Environment in South Korea has a plan to continuously expand diffuse pollution management areas until 2020. 30 In 2009, the candidate management areas for industrial sites and small watersheds had been additionally determined for conducting a feasibility study and preparing selection criteria. Relevant authorities also established the Second Comprehensive Plan for Diffuse Pollution Management (´12~´20) in a collaborative project. As a part of this plan, an improvement scheme(draft) for the criteria of determining and assessing management areas has been prepared and implemented in order to expand diffuse pollution management areas and improve the related systems. Accordingly, the order of priority needs to be set on a scientific basis so that more desperate areas for diffuse pollution management could be systematically preferred. For the diffuse pollution policy, a vulnerability analysis and a map of vulnerable areas are required. 5 Some studies (Bang et al., 1997;Yoon et al., 2007;Choi et al., 2009;Park et al., 2014) by the Ministry of Environment and the Korea Environment Corporation have been conducted so far on the nomination of diffuse pollution management area.
However, most of these studies assumed scenarios after considering significant factors and merely analyzed the scenarios to predict the results of assumptions. Consequently, determining, allocating and scoring the evaluation items and weights have become key research issues to identify vulnerable areas to non-point source pollution. To identify and prioritize vulnerable 10 areas for diffuse pollution management, the evaluation items to be made as well as the weighting and scoring methods for each item are to be importantly considered and determined. In this regard, many experts have constantly expressed the opinion that research and survey are needed to propose specific indexes quantifying the contribution rates and weights, and to determine a selection method for the vulnerability assessment (Park et al., 2013).
The degradation of water quality and aquatic ecosystem due to diffuse pollution sources is related to uncertain factors such as 15 emission characteristics of various pollutants, and climate and soil properties. Naturally, diverse solutions are being proposed by experts and interested parties, which prevent the clear policies from being firmly implemented. For efficient policy implementation, a quantitative, objective and scientific analysis needs to be conducted for factors causing diffuse pollution, and then the management areas are to be expanded. This will result in not only systematic policy implementation but also highly efficient low investment. 20 The process of the study to prioritize watersheds for diffuse pollution management in Korea is presented in Fig. 1.
Step 1 sets an evaluation framework and performs a Delphi survey for experts to determine evaluation items and weights.
Step 2 collects and quantifies data of each evaluation item for each watershed. The TOPSIS method, which is one of MCDM techniques, is applied for evaluation.
Step 3 prepares the vulnerable area map by using evaluation results, and selects major vulnerable areas. 25

Determining criteria with modified Delphi method
The Delphi method was developed by the RAND Corporation in the 1950's aimed to reduce the range of group responses and to strive for expert consensus. The Delphi method as a method for structuring a group communication process is accomplished by some feedback of individual contributions of information and knowledge, some assessment of the group judgment or view, some opportunity for individuals to revise views and some degree of anonymity in individual responses (Linstone and Turoff, 30 1975). A series of questionnaires with controlled opinion feedback is typically used for collecting and distilling knowledge from a group of experts (Rowe et al., 1991;Adler and Ziglio, 1996;Angus et al., 2003). The process that experts reply to questionnaires, subsequently receive feedback, and modify their opinion is repeated until arriving at the most reliable consensus.
The Delphi method is effective in allowing a group of individuals, as a whole, to deal with a complex problem. (Mohorjy and Aburizaiza, 1997) and has been applied in various fields such as information system, planning, environmental impact assessment, social policy and public health (Angus et al., 2003;Linstone and Turoff, 2002;Okoli and Pawlowski, 2004). Also, 5 there were several applications in studies on water resources uses and management, water quality assessment and so on (Cude, 2001;Kim et al., 2013;Lee et al., 2013;Parparove et al., 2006;Parparov and Hambright, 2007).
For successful progress, there are two important considerations. First important thing is to select experts participating in our survey. We considered that the respondents should be experts with plenty of experiences and have a high level of responsibility.
Second important thing is to reach an agreement involving experts. Making consensus will take a long time as far as that goes 10 because every experts have their own opinion that sometimes it is extremely different.
We have simplified Delphi process. The nub of modified Delphi procedure is that experts were provided detailed and concrete information for candidate criterion by organizing group. Fig. 2 explains the procedure of our modified Delphi.
Firstly, we selected a group of experts in diffuse pollution management of Korea. The experts are ones who have extensive experiences with a high level of responsibility or have carried out a lot of researches on diffuse pollution. Then we created an 15 evaluation framework with the candidates of criteria and sub-criteria after examining literature and brainstorming. This is because it is time-consuming and inefficient to reach an agreement after every expert suggests an evaluation framework. A questionnaire was prepared to obtain opinion of experts for the framework's structure, the candidates of criteria and sub-criteria.
After collecting and analyzing the judgments of a group of experts, the evaluation framework consisting of criteria and their weights was determined if the consensus of the group emerged. 20

Assessing potential risk with TOPSIS
In this study, the criteria scores were estimated by the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method of multi-criteria decision analysis methods (Fishburn, 1967;Hwang and Yoon, 1981). The TOPSIS chooses the alternative of the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution (Lai et al., 1994;Chu, 2002;Jun et al., 2011;Lee et al., 2013). 25 In addition, assessment results for all alternatives can be easily calculated and presented from multi-attribute perspective (Kim et al., 1997;Shih et al., 2007;Lee and Chung, 2007;Chung and Lee, 2009).
Here the Positive Ideal Solution (PIS) is the most vulnerable area and the Negative Ideal Solution (NIS) is the lowest vulnerable area. The TOPSIS procedure is as follows: Construct the weighted decisions matrix ( )) 30 Where is the weight of i th criterion , are built by alternative ( = 1, ⋯ , ) which are evaluated against criteria ( = 1, ⋯ , ). the standardized data of each assessment unit area.
Determine the PIS( + ) and NIS( − ) of unit area Here + = max ( ), − = min ( ) Calculate the distance from the positive ideal ( + ) and the negative ideal ( − ) solution for each alternative Calculate the optimum membership degree ( + ) 10 The priority of watersheds in terms of the need of diffuse pollution management was decided according to the criteria scores aggregated for watersheds. The priority of diffuse pollution management was represented in a map with a geographic information system (GIS).

Study Area
The MOE shall regularly survey the kinds of sources of pollution in order to ascertain the current status of water quality and aquatic ecosystems by river-system spheres of influence and shall develop the basic plan for preserving the water quality and aquatic ecosystems (WQECA Article 22 and Article 23, 1997). The spatial extent of the investigation is the whole country of Korea. 20 The river-system spheres of influence are classified by small, medium and large areas of influence. In the study, the potential risk of diffuse pollution was evaluated for 814 watersheds which are the small areas of influence and the subjects of the pollution survey.

Determination of the evaluation framework by the Delphi survey
The evaluation framework, criteria and their weight were determined by the expert's agreement through the Delphi survey. 25 The weights of multi-criteria were decided by the ranking method.
A nonpartisan expert pool was asked whether they will participate or not. Total 12 experts gave us a positive answer. They worked in the following different sectors: government-funded research institutes (8%), private engineering companies (17%), public servant (17%) and university (58%). All had doctoral-level training and most (80%) have been researching on diffuse source pollution management over 10 years.
Candidates of criteria and sub-criteria were organized based on brainstorming with literature research. Then, the draft questionnaire including candidate criteria was distributed for experts. This is the start of the first round. After we collected and analyzed the raw data from the questionnaires, we then revised the questionnaire. The modified questionnaire included the 5 analysis result of previous survey. The round processed in the same way continued until consensus emerges.
A draft framework to prioritize the watersheds by evaluating the potential risk of diffuse pollution was developed in consideration of the availability of data related to diffuse pollution in Korea and the characteristics of diffuse pollution discussed in other studies (Chon et al., 2012;Novotny, 2002;Jang et al., 2012;Jung et al., 2011;Park et al. 2010). Although the diffuse pollution is irregular, variable and indefinable and its risk varies depending on the watershed (Candela et al., 2009;10 EA, 2007;U.S.EPA, 1997), the 'source-pathway-receptor' concept is applicable and useful for the evaluation. Chon et al. to the process of generation, discharge and delivery to receiving waters of agricultural areas. In this study, pollution source, hydrologic process, and status of receiving water were employed as the groups to classify criteria. 15 The criteria of land use, activities in urban areas and agricultural areas and the sub-criteria such as population density, livestock numbers, fertilizer use, and area of different land use were selected for the pollution source; the criteria of rainfall and runoff and the sub-criteria such as annual rainfall, rainy days, drainage area, and runoff ratio for the hydrologic process, the criteria of water resource, water quality and aquatic ecosystem and the sub-criteria such as river flow, water quality of BOD, SS, and the indicator of aquatic ecosystem health for the status of receiving water. The criteria and the sub-criteria of the draft of 20 evaluation framework are as shown in Table 1.
The Delphi survey was carried out to get the approval of experts for the evaluation framework. We selected the total 13 experts with intention of participating in the survey. The experts with experience, education and training at the doctoral-level have been working for government-funded research institutes, government-affiliated organizations or universities and most of them have been involved in the diffuse source pollution management over 10 years. The experts were asked to check the structure 25 of the evaluation framework, to exclude/add the criteria and the sub-criteria and to decide their weight. At the first round, they supported the group and the criteria of the draft of evaluation framework but requested to modify some of the sub-criteria. The sub-criteria were classified by 'Acceptance', 'Reject', and 'Addition' and were revised for the second round. The feedback on the modified questionnaires was positive and the expert's consensus was built as shown in Table 2.
In this study, two types of the weight sets selected by the expert panels are used. First type is the ranking sets ( ). For a 30 given set of variables, each participant determines the ranking in the order of its highest value for each of the factors in a manner that determines and quantifies its importance. Thus, the most important factor becomes rank 1, and the next most important one is rank 2. At this point, the component may be ranked in the same order. The constructed ranks are aggregated through the conversion, with Rank 1 being converted into m-1 and Rank 2 being converted into m-2. Where m is the total number of factors. Calculate the order of these conversions by the following eq. (5) and (6).
where is the sum of transformed ranks, is the rank that was selected by the j th panel of experts and m is the total number 5 of criteria, n is the total number of expert panels.
The weights of the group, the criteria and the sub-criteria were determined by the ranks suggested by the experts. The experts judged that the pollution source (0.4853) is more important than the hydrologic process or the receiving water in the Group.
Second type of weights set is the rating sets ( ). It is a way to compare importance of criteria to distribute weights. The survey respondents will determine the weights for each criteria, but will then select values within the range given. The range 10 of weights is a continuous section, generally ranging from 0.0 to 1.0 or up to 100.0. In addition, the sum of the weights given to all variables under comparison is equal to the maximum range given. A factor equivalent to 0.0, the lowest limit of the section, is of no importance to the assessment, whereas a maximum value means that a maximum number of possible values of significance are applied.
The weights may be calculated from the following eq. (7) and (8). It is also possible to set the same value by allocating the 15 relative importance of each factor.
is the weight of criteria i, determined by Panel j.
The evaluation framework thus established can be applied flexibly in various conditions including securing of relevant data. 20 In other words, if data are insufficient or uncertain, evaluations are conducted either by removing or applying such insufficiency or uncertainty, and the evaluation results are analyzed to improve the framework. This "Adaptive Management" method is an iterative approach (Holling, 1978;Waters, 1986) that enhances the management ability by accumulating accurate understanding and knowledge of response for a target system.

Collection, quantification and standardization of evaluation data 25
(1) Data collection and quantification for each evaluation item Data such as population density, urbanization level and fertilizer use were collected from the source of either the statistics of local governments or the Statistics Korea, and the precipitation data including annual rainfall were provided by the Korea Meteorological Administration. In addition, land use map, industrial condition, combined/sanitary sewer system, livestock numbers, livestock barn area, watershed map, water resource and water quality were derived from the data surveyed by the When values of each evaluation item for 814 small watersheds were determined, some data items were not measured or missed.
As for the water quality data, if there is a water quality observatory in a watershed, the data were obtained from it, and if there is no such observatory and thus no measurement was available, data from an adjacent watershed or lake were analyzed and utilized. As for flow rate, if a small watershed consists of a single basin, the measurement of flow rate is attributable to the watershed. On the other hand, if there is another upstream small watershed, the data measure in downstream small watershed 5 cannot represent the characteristic of each small watershed. In order to improve this problem, the flow rate, rainfall, and areas of the upstream basin and small watershed were used to calculate a specific discharge and determine the flow rate of each small watershed.
where, Q is flow rate, P is rainfall and A is the basin area at the calculation point of flow rate. The subscript 1 means the 10 reference point, and the subscript 2 indicates the calculation point of flow rate.
Since many small watersheds including estuaries do not have any measurement even at the level of middle watersheds, such watersheds were left unmeasured and a low score was given without using data of adjacent small watersheds. In addition, if necessary, flow rate data of a dam were also utilized to represent runoff characteristics of small watersheds.
Collected data were spatially distributed as Fig.3. 15 (2) Standardization of evaluation items Because each dataset for evaluation items has different units and properties, standardization is required to use datasets for evaluation. The re-scaling method was adopted in the standardization process. The overall range of data was normalized to assign values between 0 and 1, as described in the equation (10). 20 = − − (10) where, is the i th standardized value, is the i th data value, is the maximum value, and is the minimum value.
However, in case the data collected are used to standardize evaluation items without modification, the standardized scores are often either biased or equalized in their range and distribution according to characteristics and types of data.
Accordingly, since it seemed to be unreasonable to apply the above equation with no modification, the data collected were 25 prioritized and the consequential order of priority was scored before the equation was used for standardization. Detailed modification process shows as table 4.

Assessing Vulnerability
The vulnerability of every small watershed to diffuse pollution was evaluated by using data and weights for each factor, and the vulnerable areas were determined based on this assessment (Fig. 4). In addition, the small watersheds were prioritized 30 again in each of 4 large watersheds, and top 30 small watersheds are illustrated in Fig. 5. This was because the pollution source management and relevant policies were organized based on the large watersheds. Both ranking and ratio methods were applied to calculate weights.
Among top 50 small watersheds in the order of priority in each large watershed, main rivers and small watersheds, which required diffuse pollution source management, were derived in each river system.
Han River basin has 3 priority control target river: downstream of Namhan River, Mid-down stream of Han River and Anseong 5 stream. Geum River basin has 4 priority control target river: midstream of Geum River, Dongjin River, Mankyeng River and Sapgyo stream.
Youngsan River basin has 2 priority control target river: Youngsan River and Sumjin River. Most vulnerable area of Nakdong River basin is in main stream of Nakdong River.
The evaluation results were analyzed in terms of effects of each evaluation factor. It turned out that if a large number of 10 livestock are reared and much fertilizer is used in a basin, the land area is wide and the public water has much soil and high SS concentration, such a watershed needs to be preferentially managed.

Conclusion
There are little studies to assess watersheds in respect of the diffuse pollution management in Korea. This study has suggested a scientific analysis method for selecting priority areas in the current diffuse pollution management system. As various 15 uncertain factors are included in assessing vulnerable areas to diffuse pollution sources, such factors need to be quantified and analyzed objectively and scientifically. The Delphi method was used to determine the vulnerability evaluation items, which included basin characteristics, pollution source and water quality, and weights for diffuse pollution, on the basis of expert opinions. Criteria and sub-criteria were allocated into three groups of pollution source, hydrologic process, and receiving water.
Based on the weights and evaluation items thus obtained, data of each item were applied, and the vulnerability to diffuse 20 pollution was assessed by the TOPSIS method. The proposed evaluation process will promote efficient policy implementation and set a foundation for scientific/clear diffuse pollution management.
In addition, this study attempted a small watershed-based analysis for more selective/intensive policy enforcement. However, it was difficult to standardize quantitatively each evaluation item, in the determination of management areas, at the level of small watershed. 25 Accordingly, a runoff model needs to be applied to improve the estimations for unmeasured areas. A vulnerability assessment system for diffuse pollution is also to be established in order to promote efficient policy enforcement. Such system should update relevant data and enable cyclic reevaluation.
Finally, this study has not reflected the current diffuse pollution management policy in the list of evaluation items. It was because the effect of the policy could not be accurately quantified. A further study will solve this problem and include the EA(Environment Agency): The unseen threat to water quality: Diffuse water pollution in England and Wales report, Bristol,

Combined/Sanitary sewer system
Estimate the sum of values with the weights 1) using the ratio of the sewage system, combined sewer system, the area without public sewage system 1) the ratio of the sewage system 0.25, the ratio of combined sewer system 0.5, the ratio of the area without public sewage system 1.0

Livestock numbers
Estimation of livestock numbers with the weights 1) by the type of livestock 1)

River improvement
Assessment score of inhabitation/waterside environment 1) 1) natural longitudinal and transverse shoal of river, width of Riverside, sediment quality, river-crossing structures, channel characteristics, embankment material, land use in inner and outer land, treatment Facilities, and etc.

Water quality BOD, TN, T-P
Calculate the sum of standardized 1) BOD, TN, TP on measuring points 1) the ratio of each water watershed to total watershed of average resource SS Standardized SS of water quality measurement site Other items (COD, Chlorophyll a, water temperature) Calculate the sum of standardized COD, Chlorophyll a, water temperature on measuring points Aquatic ecosystem

Aquatic ecosystem health
Total score of aquatic ecosystem health 1) 1) Mean of Trophic Diatom Index of attached algae, Korea Purity biotic Index (KPI) using benthic macroinvertebrates, and Index of Biological Integrity(IBI) of fish