The Ministry of Environment in the Republic of Korea has a plan to continuously expand diffuse pollution management areas until 2020. In 2009, the candidate management areas for industrial sites and small watersheds had been additionally determined for conducting a feasibility study and preparing selection criteria. Relevant authorities also established the Second Comprehensive Plan for Diffuse Pollution Management (from 2012 to 2020) in a collaborative project. As a part of this plan, an (draft) improvement scheme for the criteria of determining and assessing management areas has been prepared and implemented in order to expand diffuse pollution management areas and improve the related systems. Accordingly, the order of priority needs to be set on a scientific basis so that areas which require diffuse pollution management most urgently will receive be systematic preference. For diffuse pollution policy, a vulnerability analysis and a map of vulnerable areas are required.

Some studies (Bang et al., 1997; Yoon et al., 2007; Choi et al., 2009; Park et al., 2014) by the Ministry of Environment and the Korea Environment Corporation have been conducted so far on the nomination of diffuse pollution management areas. However, most of these studies assumed scenarios after considering significant factors and merely analyzed the scenarios to predict the results of assumptions. Consequently, determining, allocating, and scoring the evaluation items and weights have become key research issues for the identification of areas vulnerable to non-point source pollution. To identify and prioritize vulnerable areas for diffuse pollution management, the items to be evaluated as well as the weighting and scoring methods for each item should be importantly considered and determined. In this regard, many experts have constantly expressed the opinion that research and surveys are needed to propose specific indexes quantifying the contribution rates and weights, and to determine a selection method for vulnerability assessment (Park et al., 2013).

The degradation of water quality and aquatic ecosystem due to diffuse pollution sources is related to uncertain factors such as emission characteristics of various pollutants, and climate and soil properties. Naturally, diverse solutions are being proposed by experts and interested parties, which prevents clear policies from being firmly implemented. For efficient policy implementation, a quantitative, objective, and scientific analysis needs to be conducted for factors causing diffuse pollution, and then the management areas should be expanded. This will result in not only systematic policy implementation but also highly efficient, low-cost investment.

The process of the study to prioritize watersheds for diffuse pollution management in South Korea is presented in Fig. 1.

Step 1 sets an evaluation framework and performs a Delphi survey for experts to determine evaluation items and weights.

The Delphi method was developed by the RAND Corporation in the 1950s aiming to reduce the range of group responses and to strive for expert consensus. The Delphi method as a method for structuring a group communication process is accomplished by feedback of individual contributions of information and knowledge, assessment of the group judgment or view, an opportunity for individuals to revise views, and a degree of anonymity in individual responses (Linstone and Turoff, 1975). A series of questionnaires with controlled opinion feedback is typically used for collecting and distilling knowledge from a group of experts (Rowe et al., 1991; Adler and Ziglio, 1996; Angus et al., 1996). The process by which experts reply to questionnaires, subsequently receive feedback, and modify their opinion is repeated until arriving at the most reliable consensus.

The Delphi method is effective in allowing a group of individuals, as a whole, to deal with a complex problem (Mohorjy and Aburizaiza, 1997) and has been applied in various fields such as information systems, planning, environmental impact assessments, social policy, and public health (Angus et al., 1996; Linstone and Turoff, 1975; Okoli and Pawlowski, 2004). Also, there have been several applications in studies on water resource use and management, water quality assessment, and so on (Cude, 2001; Kim and Chung, 2013; Lee et al., 2013; Parparov et al., 2006; Parparov and Hambright, 2007).

For successful progress, there are two important considerations. The first is to select experts participating in our survey. We considered that the respondents should be experts with plenty of experience and with a high level of responsibility. The next important consideration is to reach an agreement involving experts. Obtaining consensus will take a long time because every expert has their own opinion, and these are sometimes extremely different.

We have simplified the Delphi process. The nub of modified Delphi procedure is that experts were provided detailed and concrete information for candidate criterion by organizing group. Figure 2 explains the procedure of our modified Delphi.

Firstly, we selected a group of experts in diffuse pollution management of South Korea. The experts are ones who have extensive experiences with a high level of responsibility or have carried out a lot of research on diffuse pollution. Then we created an evaluation framework with the candidates of criteria and sub-criteria after examining literature and brainstorming. This is because it is time-consuming and inefficient to reach an agreement after every expert suggests an evaluation framework. A questionnaire was prepared to obtain expert opinions for the framework's structure, and the candidate criteria and sub-criteria. After collecting and analyzing the judgments of a group of experts, the evaluation framework consisting of criteria and their weights was determined if the consensus of the group emerged.

In this study, the criteria scores were estimated by the TOPSIS method of multi-criteria decision analysis methods (Fishburn, 1967; Hwang and Yoon, 1981). The TOPSIS chooses the alternative of the shortest geometric distance from the positive ideal solution and the longest geometric distance from the negative ideal solution (Lai et al., 1994; Chu, 2002; Jun et al., 2011; Lee et al., 2013).

In addition, assessment results for all alternatives can be easily calculated and presented from a multi-attribute perspective (Kim et al., 1997; Shih et al., 2007; Lee and Chung, 2007; Chung and Lee, 2009).

Here the positive ideal solution (PIS) is the most vulnerable area and the negative ideal solution (NIS) is the least vulnerable area. The TOPSIS procedure is as follows:

Construct the weighted decisions matrix (vij): 1vij=wi×xij, where wi is the weight of ith criterion, xij are built by alternative Aj(j=1,…,n), which are evaluated against criteria Ci(i=1,…,m), the standardized data of each assessment unit area.

Determine the PIS(A+) and NIS(A-) of unit area: 2A+=v1+,…,vn+,3A-=v1-,…,vn-. Here vi+=max(vij), vi-=min(vij)

Calculate the distance from the positive ideal (di+) and the negative ideal (di-) solution for each alternative 4di+=∑j=1nvij-vj+21/2,i=1…,n5di-=∑j=1nvij-vj-21/2,i=1…,n. Calculate the optimum membership degree (Di+): 6D+=di-di+-di-,(i=1,…,m). The priority of watersheds in terms of the need for diffuse pollution management was decided according to the criteria scores aggregated for watersheds. The priority of diffuse pollution management was represented in a map with a geographic information system (GIS).

The MOE shall regularly survey the kinds of sources of pollution in order to ascertain the current status of water quality and aquatic ecosystems by river-system spheres of influence and shall develop the basic plan for preserving water quality and aquatic ecosystems (WQECA, 1997, Article 22 and Article 23). The spatial extent of the investigation is the whole country of South Korea.

The river-system spheres of influence are classified as small, medium, and large areas of influence. In the study, the potential risk of diffuse pollution was evaluated for 814 watersheds which are the small areas of influence and the subjects of the pollution survey.

The evaluation framework, criteria, and their weights were determined by the experts' agreement through the Delphi survey. The weights of multi-criteria were decided by the ranking method.

A nonpartisan expert pool was asked whether they would participate or not. A total of 12 experts gave us a positive answer. They worked in the following different sectors: government-funded research institutes (8 %), private engineering companies (17 %), public services (17 %), and university (58 %). All had doctoral-level training and most (80 %) have been researching diffuse source pollution management for over 10 years.

Candidates of criteria and sub-criteria were organized based on brainstorming with literature research. Then, the draft questionnaire including candidate criteria was distributed for experts. This was the start of the first round. After we collected and analyzed the raw data from the questionnaires, we then revised the questionnaire. The modified questionnaire included the analysis result of previous survey. The round proceeds in the same way, continued until consensus emerges.

A draft framework to prioritize the watersheds by evaluating the potential risk of diffuse pollution was developed in consideration of the availability of data related to diffuse pollution in South Korea and the characteristics of diffuse pollution discussed in other studies (Chun et al., 2012; Novotny, 2002; Jang et al., 2012; Jung et al., 2011; Park et al. 2010). Although the diffuse pollution is irregular, variable, and indefinable and its risk varies depending on the watershed (Candela et al., 2009; EA, 2007; US EPA, 1997), the “source–pathway–receptor” concept is applicable and useful for the evaluation. Chun et al. (2012) defined criteria of activities and land-use representing pollution sources, rainfall, and runoff characteristics, and physical, chemical, and ecological status of the receptor adopting the concept. Similarly, Jang et al. (2012) used the characteristics related to the process of generation, discharge, and delivery to the receiving waters of agricultural areas. In this study, the pollution sources, hydrologic processes, and status of receiving water were employed as the groups used to classify the criteria.

The criteria of land use, activities in urban areas and agricultural areas, and the sub-criteria such as population density, livestock numbers, fertilizer use, and area of different land use were selected for the pollution source; the criteria of rainfall and runoff and sub-criteria such as annual rainfall, rainy days, drainage area, and runoff ratio for the hydrologic processes; the criteria of water resources, water quality, and aquatic ecosystems and the sub-criteria such as river flow, water quality based on biochemical oxygen demand (BOD), suspended solids (SS), and the indicator of aquatic ecosystem health were selected for the status of the receiving water. The criteria and the sub-criteria of the draft of evaluation framework are as shown in Table 1.

The Delphi survey was carried out to get the approval of experts for the evaluation framework. We selected a total of 13 experts who showed interest in participating in the survey. The experts, with experience, education, and training at the doctoral-level, work for government-funded research institutes, government-affiliated organizations, or universities, and most of them have been involved in diffuse source pollution management for over 10 years. The experts were asked to check the structure of the evaluation framework, to exclude/add the criteria and the sub-criteria, and to decide their weight. At the first round, they supported the group and the criteria of the draft of evaluation framework but requested to modify some of the sub-criteria. The sub-criteria were classified with “Acceptance”, “Rejection”, and “Addition” and were revised for the second round. The feedback on the modified questionnaires was positive and the experts' consensus was built as shown in Table 2.

In this study, two types of the weight sets selected by the expert panels are used. The first type is the ranking sets (wirank). For a given set of variables, each participant determines the ranking in the order of its highest value for each of the factors in a manner that determines and quantifies its importance. Thus, the most important factor becomes rank 1, and the next most important one is rank 2. At this point, the component may be ranked in the same order. The constructed ranks are aggregated through the conversion, with Rank 1 being converted into m-1 and Rank 2 being converted into m-2, where m is the total number of factors. Calculate the order of these conversions by Eqs. (5) and (6). 7Ri=∑j=1nRij,8wirank=Ri/∑i=1mRi, where Ri is the sum of transformed ranks, Rij is the rank that was selected by the jth panel of experts, and m is the total number of criteria, n is the total number of expert panels.

The weights of the group, the criteria, and the sub-criteria were determined by the ranks suggested by the experts. The experts judged that the pollution source (0.4853) is more important than the hydrologic process or the receiving water in the group.

The second type of weights set is the rating sets (wirate). It is a way to compare importance of criteria to distribute weights. The survey respondents will determine the weights for each criteria, but will then select values within the range given. The range of weights is a continuous section, generally ranging from 0.0 to 1.0 or up to 100.0. In addition, the sum of the weights given to all variables under comparison is equal to the maximum range given. A factor equivalent to 0.0, the lowest limit of the section, is of no importance to the assessment, whereas a maximum value means that a maximum number of possible values of significance are applied.

The weights may be calculated from Eqs. (7) and (8). It is also possible to set the same value by allocating the relative importance of each factor. 9wij=pij/∑i=1mpij,10wirate=∑j=1nwij/∑j=1n∑i=1mwij, where pij is the weight of criteria i, as determined by panel j. Established weights are shown in Table 3.

The evaluation framework thus established can be applied flexibly in various conditions including securing of relevant data. In other words, if data are insufficient or uncertain, evaluations are conducted either by removing or applying such insufficiency or uncertainty, and the evaluation results are analyzed to improve the framework. This “adaptive management” method is an iterative approach (Holling, 1978; Walters, 1986) that enhances management ability by accumulating an accurate understanding and knowledge of response for a target system.

The vulnerability of every small watershed to diffuse pollution was evaluated by using data and weights for each factor, and the vulnerable areas were determined based on this assessment (Fig. 4). In addition, the small watersheds were prioritized again as part of each of four large watersheds, and top (more vulnerable) 30 small watersheds (of these large watersheds) are illustrated in Fig. 5. This was because the pollution source management and relevant policies were organized based on the large watersheds. Both ranking and ratio methods were applied to calculate weights.

Among top 50 small watersheds in the order of priority in each large watershed, main rivers and small watersheds, which required diffuse pollution source management, were derived in each river system.

The Han River basin has three priority control target rivers: downstream of Namhan River, middle of the Han River, and Anseong Stream. The Geum River basin has four priority control target rivers: midstream of the Geum River, Dongjin River, Mankyeng River, and Sapgyo Stream.

Youngsan River basin has two priority control target rivers: Youngsan River and Sumjin River. The most vulnerable area of the Nakdong River basin is in main stream of Nakdong River.

The evaluation results were analyzed in terms of effects of each evaluation factor. It turned out that if a large number of livestock are reared and much fertilizer is used in a basin, the land area is wide, and the public water has much soil and a high SS concentration, such a watershed needs to be preferentially managed.