Determination of Heavy Rain Damage-Triggering Rainfall Criteria Based on Data Mining

Heavy rainfall occurs over the Korean peninsula mainly because of typhoons and a localized heavy rainfall, leading to severe flooding and landslide risk. KMA (Korean Meteorological Administration) has the criteria for issuing a Heavy Rain Advisory (HRA) over the peninsula even though each region or local government has different conditions in capability of 10 disaster prevention (CDP) and different characteristics in rainfall and heavy rain damage. Therefore, the aim of this study is to suggest the methodology for the determination of Heavy rain Damage-Triggering Rainfall Criteria (HD-TRC) that HRA can be issued in each region. The study regions are local governments in Gyeonggi-province, Seoul-city, and Incheon-city in Korea. HD-TRC can be determined based on rainfall and heavy rain damage data. The data from 2005 to 2018 are collected and then the data for flood or rainy season from June to September are extracted. The rainfall data is provided in KMA and heavy rain 15 damage data during disaster periods (DPs) can be obtained from the statistical yearbook of natural disaster (SYND) published by MOIS (Minstry of Interior and Safety) every year. Training set of 2005 to 2014 is used for obtaining HD-TRC and test set of 2015 to 2018 is used for evaluating three criteria of HD-TRC, Advanced HD-TRC, and HRA. Analysis for determining the best criteria is performed through data mining processes as follows: (1) Maximum rainfalls in durations of 1 to 24-hr (X1) and antecedent rainfalls of 1 to 7-day (X2) are obtained and used as independent variables. Heavy rain damage data are divided 20 into damage day (“1”) and no damage day (“0”) used as dependent variables (Y). Principal component analysis (PCA) is performed and PCs (principal components) are obtained as PC. X1 and PC. X2 for independent variables. Then Risk Index (RI) is defined as PC. X1 + PC. X2 and RIs become the candidates for HD-TRC. The predicted damage(?̂?) is obtained based on RIs and confusion matrix is constructed then the best HD-TRC is determined through the evaluation of classification performance. (2) However, ‘abnormal days’ (ADs) in a DP that the damage is occurred exists. The ADs mean the days which we do not 25 have rainfall or have small rainfall amount during DP. Say, ADs have too small rainfall to damage even during DP. The ADs are defined as days below rainfall of 20mm and 5 cases of ADs are also defined as 0, 0~5, 0~10, 0~15, and 0~20mm in this study. We count total days in all the DPs and in ADs for a case. The ratio of ADs to total days during DPs could be the occurrence probability or prior probability (PP) of ADs for a case and 5 PPs are obtained. Also, the average AD for each case can be obtained and defined as risk range (RR). Then we define Advanced HD-TRC using MCS (Monte Carlo Simulation) 30 linked with PP, RR, and from HD-TRC for each case. Therefore, HD-TRC is determined based on RI and Advanced HDhttps://doi.org/10.5194/nhess-2020-295 Preprint. Discussion started: 9 October 2020 c © Author(s) 2020. CC BY 4.0 License.

TRC for each case based on PP and RR. Finally, three criteria of HD-TRC, Advanced HD-TRC, and HRA are compared based on performance evaluation by test set. As a result, Advanced HD-TRC shows the best performance and so the suggested methodology can be used for regional heavy rain damage warning information.

Introduction 35
The frequency of natural disaster and damage scale are increasing trend over the world due to climate change and rapid urbanization. In the last decade, total damage by natural disaster was 3.4 trillion KRW (Korean Won) (USD 2.83 billion) in Korea. Especially, the damage by heavy rain was 3.4 trillion KRW (USD 1.25 billion) (Ministry of the Interior and Safety, MOIS, 2018). Therefore, the structural measures such as dam, levee or dike, and flood control channel have been constructed while the nonstructural measures of natural or heavy rain disaster forecasting and warning system have been also used for the 40 damage reduction. However, in recent, people are aware of the importance of the conservation of nature and the value of ecosystem and so these days, our preference is for the nonstructural measures rather than the structural measures. Forecasting and early warning system for flood or heavy rain damage is one of examples of the nonstructural measures which can give early warning to the affected people. This can provide information for preparing to match urgent needs with available resources and for emergency response according to an action plan then it can considerably reduce heavy rain loss and damage, and the 45 loss of human lives.
However, the accuracy of forecasting and warning system is required in two aspects. First is about the accuracy of meteorological information which is forecasted by various models and the other is the accuracy of warning criteria. Weather forecasting has been studied for a long time and various models have been used for the future prospects of meteorological information considering climate change (Kannan et al., 2010;Abbot et al, 2012;Mekanik et al, 2013;Abbot et al., 2014;Kim 50 et al., 2014). Many researches have been also performed for the evaluation of flood and drought risks by using the future meteorological information (Kay et al., 2006;Dankers et al., 2008;Robert et al., 2012;Zbigniew et al., 2013). But the accuracy of long-term forecasting for the future can not be guaranteed because many uncertainties are involved in observations and information and so the study has concentrated on short-term forecasting of meteorological information using various statistical methods such as regression model, neural network model, machine learning, and so on (Chau et al., 2010;El-Shafie et al., 55 2011;Abhishek et al., 2012;Kim et al., 2013;Cramer et al., 2017;Mishra et al., 2018;Chatterjee et al., 2018;Luitel et al., 2018). While the researchers have interested in the studies for meteorological forecasting, there have not been enough studies to develop the warning criteria. The early warning system for landslide has warning criteria according to earthquake intensity but there is no warning criteria for heavy rainfall in Korea. Actually, the landslide and failure of slope are related to the localized heavy rainfall and typhoons 60 mostly occurred in June to September. Therefore, the previous studies said that the warning criteria of landslide should be determined according to heavy rainfall and so the statistical, empirical, and satellite methods have been applied to determine the critical or threshold value of landslide-triggering heavy rainfall (Glade et al., 2000;Piciullo et al., 2016;Rossi et al., 2017;https://doi.org/10.5194/nhess-2020-295 Preprint. Discussion started: 9 October 2020 c Author(s) 2020. CC BY 4.0 License. Lee et al., 2017). Also, the early warning for flood is determined by threshold value of flood discharge obtained from flood forecasting based on hydrologic rainfall-runoff model (Beguería et al., 2006;Montesarchio et al., 2009;Alfieri et al., 2015;65 Forestieri et al., 2016;Miao et al., 2016;Sieq et al, 2017;Zhai et al., 2018;Sairam, 2019). However, the rainfall-runoff model can be used just for a specific region around the river or a river basin. There are statistical methods of rainfall ensemble and the relationship of IDF (Intensity-Duration-Frequency) curve and the return period for determination of critical rainfall (Kim et al., 2011;Bezak et al., 2015;Yang et al., 2016). The critical rainfall for improvement of Heavy Rain Advisory (HRA) or Heavy Rain Warning (HRW) is determined by the occurrence probability of damage in each rainfall duration obtained by 70 Bayesian theory (Park et al., 2014;Montesarchio et al., 2015;Song et al., 2016;Cho et al., 2018;Jeong et al., 2017;Lopez et al, 2017;Choi et al., 2018;Lee et al., 2018;Kim et al., 2018). The previous studies have mostly suggested the criteria based on the specific rainfall intensity and this can explain for the localized heavy rainall but it is difficult to explain for relatively small amount of rainfall having long duration. In addition, the damage day was not considered in most of previous studies, and among them, there was no study considering the uncertainty that damage could occur even in small rainfall. 75 Therefore, this study considers rainfall intensities of 24 durations and antecedent rainfalls of 1 to 7 days for explanation of various rainfall events. Also this study consider the uncertainty of ADs additionally. The purposes of the study are to understand the regional rainfall and heavy rain damage characteristics based on data mining and to suggest Heavy rain Damage-Triggering Rainfall Criteria (HD-TRC) for each region or local government. Then we are going to propose Advanced HD-TRC through considering uncertainty of ADs stochastically. 80

Warning Criteria for Heavy Rain
Early warning system has been studied and utilized in most of the countries to mitigate natural disaster risk. Especially, heavy rain warnings are issued with the criteria when heavy rainfall is expected to cause serious damage, such as severe flooding 85 and/or landslides. As can be seen in Table 1, heavy rain advisory and warning are announced by soil water index (SWI) and heavy rainfall intensity of the best segmentalized administrative district by JAM (Japan Meteorological Agency) in Japan.
SWI is for the stored water quantity in soil and higher SWI describes higher disaster risk. Hong Kong Observatory (HKO) is operating early warning system of Amber, Red, and Black based on heavy rainfall intensity. Canada (http://www.weather.gc.ca) divided the region into three groups and is utilizing the early warning system for heavy rainfall. HRA and HRW are issued by 90 KMA when the forecasted heavy rainfall intensity is above the criteria. Table 1.
Japan, Hong Kong, and Canada are considering the regional characteristics for HRA or HRW while the same criteria over the Korean peninsular without consideration of local properties has been used for issuing HRA and HRW in Korea. In reality, each local government or region has different Capability of Disaster Prevention (CDP). When the same criteria for HRA is applied to A region having lower CDP and B region higher CDP, the damage may be occurred in A region but not in B region.
That is to say, A region can have damage even though HRA is not issued but B region does not have damage even though HRA is issued. If the regional characteristics are not properly reflected for HRA, this causes people to lose the trust for the criteria of HRA. Therefore, we may need the regional criteria of HRA which can consider heavy rainfall, heavy rain damage, and CDP characteristics of each local government. To do this, the methodology for derivation of Heavy rain Damage-100 Triggering Rainfall Criteria (HD-TRC) that means the critical or threshold rainfall criteria in each region or local government which the damage can be occurred are suggested in this study.

Statistical Yearbook of Natural Disaster
The SYND (statistical yearbook of natural disaster) which records the data related to the damage occurred due to natural 105 disaster is published by MOIS (Korean Ministry of the Interior and Safety) every year from 1985 to present. It is categorizing the damage data according to meteorological hazards such as typhoon, heavy rainfall, heavy snowfall, strong wind, ocean wave and so on. Damage data is also organized in SYND for each region such as si (city), gun (county), and gu (district).
MOIS defines Disaster Period(DP) in SYND as HRA period issued by KMA and the damage is recorded for casualty, victims, public facility, and private facility during DP (see Table 2). When a rainfall event is moving over the regions, the event affects 110 from a region that the first damage is occurred to the final damaged region and this moving period of an event is DP. However, there exists a certain region of no rainfall even during DP and so the damage could not be occurred in the region. Table 2 is describing an example of heavy rain damage occurred from August 26, 2018 to September 1, 2018 for Nowondistrict in Seoul city and this is recorded in SYND published in 2019. There are the damage items, DP in the top of right side, and total damage in the bottom of right side in SYND. Total damage was 317.94 KRW in millions and KRW is KoRean Won 115 (1USD=KRW/1,200). Therefore, all the damage and DP over the country are recorded in SYND and this study collected the heavy rain damage data and DP from 2005 to 2018 for local governments in Gyeonggi-province, Seoul-city, and Incheon-city (see a section 5.1). The damage data and the damage days during DPs are collected for the study regions and Fig. 1 is showing the damage and 120 damage days in all of the study regions (66 districts). The red line represents the no. of damage day and green bar is for the damage. Annual mean damage was about USD 38 million and the number of annual mean damage was 348. The biggest damage and damage days or damage frequency were occurred in 2011 and the damage frequency in 2006 is similar with 2011 but the damage was relatively smaller than that of 2011. The damages and damage frequencies in 2014 and 2015 were very small and so we can know that it is describing drought years. https://doi.org/10.5194/nhess-2020-295 Preprint. Discussion started: 9 October 2020 c Author(s) 2020. CC BY 4.0 License.

Rainfall Data Collection
Rainfall data is observed and managed by KMA, MOLIT (Ministry of Land, Infrastructure and Transport), and K-Water in Korea. Especially, the relatively long and reliable rainfall data record can be obtained from KMA. ASOS (Automated Surface Observing System) and AWS (Automatic Weather System) are used for measuring meteorological data including rainfall. The 130 rainfall data has been observed from 1970s by ASOS which has 101 stations in South Korea while the data observed from 2000s by AWS which has 501 stations. Therefore, this study uses the record of AWS which has more densely distributed stations and Thiessen polygon method is used for spatial distribution of the rainfall. The hourly rainfall data by AWS is collected from 2005 to 2018 and it is transformed to mean aerial precipitation of each region. Finally, this study collects maximum rainfalls in durations of 1 to 24-hr and antecedent rainfalls of 1 to 7-day from hourly rainfalls for the administrative 135 districts of each city, county, and district in Gyeonggi-province, Seoul-city, and Incheon-city, Korea.

Data Mining
Data mining is the process to find the patterns and characteristics in large data sets involving methods at intersection of statistics, 140 machine learning, and database system. It is not a simple technique and also called as KDD (knowledge-discovery in databases) (Hand, 2007;Olson, 2008). The overall processes for data mining are explained as follows (Hastie, 2009;Frank, 2011).

(1) Data Extraction
First, it is necessary to set the purpose of solving the problem. Based on the purpose of doing the analysis and deriving data 145 characteristics, the selection and collection of raw data will follow.

(2) Data Preprocessing
Data preprocessing is the second step in data mining technique. This is used to transform the raw data into a useful and efficient format. The phrase "garbage in, garbage out" is particularly applicable to data mining projects. If one use an incorrect data, 150 the obtained result will definitely be incorrect. Thus data preprocessing refers to the process of supplementing raw data such as filling the missing data and removing the redundant data.

(3) Data Transformation
In order to utilize a meaningful data analysis, deriving a representative characteristic or sample data is deemed important. 155 Representative characteristic or sample data are converted from a preprocessed data for analysis.

(4) Data Analysis
Data mining can be divided into three categories according to its purpose. The first category is association analysis, while the second category is classification. The last one is the regression prediction, wherein various analysis techniques are being used 160 such as artificial neural networks (ANN), decision trees, and random forests. Algorithms uniquely developed by researchers can also be used for analysis

(5) Evaluation and Interpretation
The final step is to evaluate and interpret the analyzed results. Whether patterns and characteristics are suitable are judged 165 through this process.
In this study, we tried to solve the classification problem related to prediction. The researchers performed data mining to derive the rain threshold which can forecast heavy rain damage from a large rainfall data. In the case of Step 4 of the data mining procedure, the most common classification techniques such as decision trees and random forest are widely used. However, this 170 study proposes a new algorithm for deriving rain threshold for intuitive understanding.

Monte Carlo simulation
In general, deterministic model can find an exact solution, while stochastic model maybe can not. In this case, a series of random numbers can be repeatedly generated and simulated to find the answer to the approximation (Kroese, 2014). This 175 method is called Monte Carlo simulation (MCS). MCS performs risk analysis by building models of possible results by substituting a range of a probability distribution for any factor with inherent uncertainty. Thus, MCS is useful for modeling with uncertainty or for problems with mathematically complex conditions (Nabian, 2017).
As an example, MCS can be used as a process for calculating pi(the ratio of the circumference of a circle to its diameter).
When calculating the area of a circle represented by 2 + 2 = 1 , the circle is completely contained within a square space 180 with four in area. When a random number that satisfies this conditions (−1 ≤ ≤ 1 and −1 ≤ ≤ 1) is generated in this space, the area of the circle can be roughly calculated by multiplying the ratio of the number of random numbers in the circle by the total area of 4. If the number of randomly sampled random numbers increases, more accurate calculation results can be obtained. However, there may be some errors due to the stochastic method. In this study, the MCS was is used to express the uncertainty of Ads (abnormal days) which the researchers have referred to as days without rainfall or have small rainfall 185 amount during DP (Disaster period). Then, we examine the improvement of classification performance when the uncertainty is considered for HD-TRC.

Principal Component Analysis
Principal component analysis is an analysis that derive new variables of small dimensions using a covariance matrix or a 190 correlation matrix. The new variables are then called Principal Components and are not correlated with each other (Hotelling, 1933). In this study, the average vector for the set of independent variables = ( 1 , 2 , ⋯ , ) is designated as and the covariance matrix is Σ, with the diagonal element of Σ is the variance 1 2 , 2 2 , ⋯ , 2 of each independent variable. The number of eigenvalues of the covariance matrix Σ is , with descending order of eigenvalues called as λ 1 , λ 2 , ⋯ , λ . Also eigenvectors corresponding to each eigenvalue are called 1 , 2 , ⋯ , . The principal components are linear combinations with eigenvectors 195 as coefficients (Wold et al., 1987). Covariance Matrix of principal component( = ( 1 , 2 , ⋯ , )) are shown in Eq.(1) by characteristics of eigenvalue and eigenvector.
The eigenvalues are sorted by size, which means that some variances of the principal components may account for most of the sum of the variances of all the variables. Thus the variance of the entire independent variable set = ( 1 , 2 , ⋯ , ) can be 200 well explained by the variance of the principal components 1 , 2 , ⋯ , of the minority of (Jolliffe, 2011).
In this study, principal component analysis was considered for the following reasons. First is that the heavy rain damage may be caused by short-term heavy rains, while small amount of rainfall with long-term may continue to cause damage. This study considers all of rainfall variables including the maximum rainfalls in durations of 1 to 24hr ( 1 ) and the antecedent rainfalls of 1 to 7days ( 2 ) in order to explain the damage caused by various types of rainfall events. However, because this study 205 considers many rainfall variables, it is difficult to determine the criteria as a representative rainfall threshold. Therefore, we tries to reduce all the variables into one principal component ( ) which can represent both types of rainfall variables. Then, HD-TRC can be defined using s.

Evaluation of Classification Performance
This study uses "1" for heavy rain damage days, and "0" for no damage days. In order to evaluate the performance in 210 classification problem, the confusion matrix should be calculated (Fawcett, 2006). The confusion matrix is explained in Table   3 which has Positive(P) for "1" and Negative(N) for "0". If the predicted value(̂ ) is 1 and the observed value( ) is 1, it is called as True Positive(TP), and if ̂ is 0 and is 0, it is True Negative(TN). In addition, if ̂ is 1 and is 0, it is False Positive(FP), and if ̂ is 0 and is 1, it is False Negative(FN). Table 3.

215
The performance evaluation indicators through confusion matrix can be defined and calculated (Table 4). These indicators are Accuracy, Error Rate, Sensitivity, Precision, and Specificity then, these can be used according to the purpose of the analysis (Powers, 2011). There are 10,000 data. Say, P ("1") is 1,000 and N ("0") is 9,000. This has imbalanced data structure. If we predict 10,000 data 220 as 0, accuracy is 90%. because N is 9,000. However, sensitivity is 0%. Therefore, it is difficult to evaluate classification performance when one indicator is used and so combinations of indicators such as AUC, ROC curve, and F1-score should be used.
The ROC (Receiver operating characteristics) curve is created by plotting the Sensitivity against the 1-Specificity at various threshold settings. The ROC curve is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its 225 discrimination threshold is varied. The area under the curve (AUC) of the ROC curve is the criterion for verification. Fig. 2 shows the concept of ROC curve and AUC. The AUC values are varied from 0.5 to 1 and closer to 1 means better classification performance. If we see Fig 2, the yellow polygon with AUC=0.95 has bigger area than the blue polygon with AUC=0.50. Due to its greater advantage, AUC is widely used as a representative performance evaluation indicator for comparing the absolute predictive performance (Choi, 2018). In general, it is appropriate to use AUC for a performance indicator of a balanced data. 230 However, when the data is imbalanced, it is difficult to evaluate the performance correctly using AUC.
F1-score is a performance evaluation indicator that can be more accurately evaluated when the data is unbalanced in binary classification. As indicated by Sasaki, 2007, F1 score shown in Eq. 2, can be calculated through the harmonic mean of the precision and recall.

235
is generally defined as 1 in F-Score, and thus, it is called F1-Score. In order to verify if F1-score is appropriate as a performance measure in the imbalanced data, Saito (2015) applied several evaluation indicators in both balanced data and imbalanced data. As a result, most performance measure have shown equal performance in the balanced data (P : 1,000, N : 240 1,000) and imbalanced data (P : 1000, N : 10,000). However it was identified that the classification performance of Precision-Recall Curve has changed in imbalanced data. Therefore, this study uses F1-score which includes precision and recall as the performance evaluation indicator.

Step 1 : Dependent and independent variables
Heavy rain damage and rainfall data are collected from MOIS and KMA then dependent and independent variables are determined as shown in Fig. 3 and Table 5. The procedure in Fig. 3 is explained as follows: ① Daily damage data obtained during DP is divided into two groups of damage day and no damage day. Damage day is 250 represented as 1 and no damage day as 0. These 1 and 0 become dependent variable, (also, see Fig. 8).
② Hourly rainfall data collected from KMA is transformed to mean aerial rainfall by Thiessen polygon method.
Then, maximum rainfalls in durations of 1 to 24-hr( 1 ) and antecedent rainfalls of 1 to 7-day( 2 ) are obtained and these become independent variables of 1 and 2 .
③ Thus, we establish dependent variable and independent variables = 1 + 2 . Also, the variables are divided into 255 training set and test set.

Step 2 : Rain index and performance test
The best HD-TRC is determined in this step by performance test for training set. Its procedure is explained in Fig. 4 according 260 to following steps: ③ All the RIs in training set become the candidates of HD-TRC. If RI at the point of time we want to predict is greater than 265 RI which is a candidate of HD-TRC, we predict the damage will be occurred. Then the confusion matrix for Obs. ( ) and Pred. (̂) is constructed and the classification performance test is conducted by F1-score for the evaluation of RI and the determination of HD-TRC. The RI having the best performance is determined as HD-TRC of the target region.

Analysis Procedure for Advanced HD-TRC
DP is defined as the period that heavy rain damage is occurred in SYND, that is to say, the period of HRA or HRW. Abnormal days(ADs) is also defined as the days which we do not have rainfall or have small rainfall amount or depth during DP(see ① and ④ of Fig. 5). If ADs are included during DP, the prediction error of damage occurrence can be leaded. Therefore, this section defines Advanced HD-TRC considering uncertainty of ADs as explained in Fig. 5 as follows: 275 ①, ②, ③ ADs of 5 cases are defined according to rainfall depth from 0 to 20mm during DPs. ADs of Case 1 is for no rainfall days during DPs, Case 2 is for rainfall days of 0~5mm, and Case 3 ~ 5 for rainfall days of each rainfall depth in ①. The https://doi.org/10.5194/nhess-2020-295 Preprint. Discussion started: 9 October 2020 c Author(s) 2020. CC BY 4.0 License. number of ADs during DPs is counted for each case and total number of days for all DPs ( ) is also counted. Then the ratio of / is defined as Prior Probability (PP). 280 ④, ⑤ ADs for the same DPs of each case are obtained and ADs in DPs are averaged for each case. The averaged AD for each case can be rounded as an integer value. This averaged AD value is defined as Risk Range (RR). Here, PP is the occurrence probability of AD and RR is the occurrence range of AD.
⑥ We can know DP if the heavy rain damage was already occurred but we can not know DP at the starting point of time for the damage prediction. Therefore, the damage is predicted by assuming the damage will be occurred when the bigger 285 rainfall than HD-TRC is produced and we consider the probability and the range that AD is included. The random number which is the probability is generated for the days corresponding to RR. The generated probability by MSC is compared with PP then the predicted damage occurrence is determined (also, see a section 5.4 and Fig. 14). Therefore, Advanced HD-TRC is defined according to PP and RR estimated in each Case.

Comparative Analysis for Determination of the Best Criteria
This section evaluates three criteria of HD-TRC, Advanced HD-TRC, and HRA and the best one is selected. the evaluation procedure is explained as follows (Fig. 6).
① HD-TRC, Advanced HD-TRC, and Heavy rain advisory(HRA) are applied to test set and we calculate performance indicators such as F1-Score and AUROC by confusion matrix. 295 ② Then the best criteria having the best performance is determined.

Study Region 300
The methodology suggested in this study is applied to the local governments of the metropolitan area. Say, Gyeonggi-province has 25 local governments or districts, Seoul-city 31, and Incheon-city 10 (Fig. 7). The study investigated the damage history for each region and district then obtained the frequency and the average frequency of heavy rain damage for each region and administrative district (Table 6). However, the damage history is not enough to suggest HD-TRCs for whole local governments or districts of city, county, and district. Therefore, this study combine all the data of local governments for Gyeonggi-province, 305 Seoul-city, and Incheon-city and suggests HD-TRCs for three regions.  https://doi.org/10.5194/nhess-2020-295 Preprint. Discussion started: 9 October 2020 c Author(s) 2020. CC BY 4.0 License.

Dependent variable 310
As mentioned in a section 4.1.1, daily heavy rain damage data during DP is obtained by SYND from 2005 to 2018. Then the dependent variable, Y of "1" and "0" for damage day and no damage day is obtained and this is explained in Fig. 8. Blue color represents damage days of DP and other days has no damage.

Independent variable 315
There are 66 administrative districts and 103 stations of AWS in three regions of Gyeonggi-province, Seoul-city, and Incheoncity. About 4.7 stations affect the each district and hourly rainfalls from 103 stations are collected from 2005 to 2018. Fig. 9 shows Thiessen polygon for the estimation of mean aerial rainfall and daily rainfall variable is obtained for each district. Then, the independent variables of maximum rainfalls in durations of 1 to 24-hr ( 1 ) and antecedent rainfalls of 1 to 7-day ( 2 ) are obtained and the variables are grouped into two sets of training set (2005 to 2014) and test set (2015 to 2018). 320 generally in average value while the number of day showed relatively higher value. This means that the small rainfall duration lasted long even annual rainfall was average value and the rain damage was frequently occurred. Therefore, mostly, the heavy 330 rain damage is increased as the rainfall is increased and so understanding damage occurrence based on the heavy rainfall is needed for HD-TRC. Because the number of damage days are much smaller than that of no damage days, it is very difficult to understand the proper 335 HD-TRC by using all the damage data. The study examines the monthly damage history to see the damage period during 2005 to 2018 and finds the damage has been occurred mainly during rainy days in rainy season of June to September (Fig. 11).

340
As we can see in Table 7, the number of damage and no damage days for rainy season are smaller than that for whole years (2005 to 2014). Especially, the number of no damage days "0" is greatly reduced but "1" is a little reduced. Say, the number of "1" is reduced by 15% but "0" which we do not need by 90% and so if we just consider rainy season the analysis becomes more easier. 345

Rain Index
There are too many independent variables to suggest them as HD-TRC and so principal component analysis (PCA) is used for deriving the principal components of . 1 from maximum rainfalls of 1 to 24-hr duration and of . 2 from antecedent rainfalls of 1 to 7-day. Then rain index (RI) is defined as following equation of (3). 350 In general, when the number of principal component is determined, the cumulative variance of each variable should be over 80% for minimizing information loss and reducing the variables. As a result, we identify the cumulative variances of . 1 355 and . 2 are over 80% (Table 8) and this means there is no problem to reduce the variables as two principal components.
That is to say, . 1 and . 2 are estimated and RIs are also obtained for each region. In training set, all RIs can be used for examining daily damage occurrence and regarded as the candidates for the determination of HD-TRC.

Determination of HD-TRC by Rain Index
The daily damage data in training set is classified as "0" and "1" for the construction of confusion matrix and evaluation indices in Table 4 are calculated for the classification performance. For an example, RI 1 becomes a criteria candidate and other RIs are compared to RI 1 then the results of comparison will be the predicted damage occurrence (̂). Next RI 2 will be another criteria candidate then others are compared to RI 2 for obtaining the other predicted damage occurrence. In this way, the 365 predicted damage occurrence is estimated and then the confusion matrix is obtained based on and ̂.
The evaluation indices such as Sensitivity, Specificity, and Precision are calculated from the confusion matrix.

Figure 12
and then the performance measures of F1-score by Sensitivity and Precision, and AUC by Sensitivity and Specificity are estimated. There are the number of data for "1" and "0" in each region. Total epoch of Gyeonggi-province has 18,055, Seoulcity 13,295, and Incheon-city 5,069. Here, epoch means that all the criteria candidates are evaluated as much as the number of data. The criteria candidate which has the largest F1-score becomes HD-TRC. Fig. 12 is showing the F1-score in all of epoch on each region. Also the red points are expressed criteria candidates which have the largest F1-score. Therefore, HD-TRCs are 375 determined based on F1-score for the regions (Table 9).  Fig. 13 is showing RIs, the determined HD-TRC, and the predicted damage occurrence on training set for each region. Red is for observed damage day, blue is for observed no damage day, triangle is for predicted damage day, and cross is for predicted 380 no damage day. Therefore, the classification can be obtained. That is, red triangle is for observed and predicted damage days (TP), blue triangle is for observed no damage day and predicted damage day (FP), red cross is for observed damage day and predicted no damage day (FN), and blue cross is for observed and predicted no damage days (TN). The black line is for the determined HD-TRC for each region. The confusion matrix is constructed by the classification results (Table 10).

Figure 13
385 RIs above the line of a determined HD-TRC in Figure 14 mean the predicted damage occurrence represented by triangle and RIs below the HD-TRC mean the predicted no damage occurrence by cross. The HD-TRC for each region is determined based on the best classification performance which has maximum F1-score. However, if we see Table 10, there are still the number of FN and FP, and also it can be seen in red cross(FN) and blue triangle(FP). FN still includes ADs and FP includes no damage 390 days even though heavy rainfall was occurred. Therefore, this study considers ADs in FN to improve the predictability or classification performance and analyzes ADs in a section 5.4. The improved HD-TRC is called Advanced HD-TRC. However, FP is not reconsidered here. The reasons why no damage, even during heavy rainfall, exists are different regional characteristics such as CDP, urbanization, resilience, and so on.

Determination of Advanced HD-TRC by Risk Range
This study defines ADs as below daily rainfall depth of 20 mm during DP and 5 cases as 0, 0 to 5, 0 to 10, 0 to 15, and 0 to 20mm in Table 11. Total number of days for DPs (M) and Total number of Ads (N) are counted (see ② and ④ of Fig. 5).
Then prior probability (PP) and risk range (RR) are also defined in Table 11 and Figure 6. PP is the probability that AD is 400 occurred and RR is the range that AD is occurred as explained in a section 4.2. We use all the raw data without preprocessing to identify more exact PP and RR in training set in this section while we used the data of June to September with preprocessing for HD-TRC in section 5.3.1.

405
Actually we do not know AD and DP for the future but we know PP and RR from observations. Therefore the study applies MCS for probabilistically obtaining the final predicted damage occurrence, ̂ by using PP and RR. An example is described for the case 2 in Table 11. Case 2 is for daily rainfall of 0 to 5mm, for PP = 0.3198(about 0.32), and RR = 2days obtained from training set. Fig. 14 is analysis procedure for an example. The first column is for row number, and second is for the damage occurrence, ̂ which can be predicted by the comparison of RI from test set with the determined HD-TRC in training set. Third 410 is for the generated probability(P) which AD can be occurred in test set and AD is corresponding to RR. The last column is for the final predicted damage occurrence, ′ obtained by the comparison of PP and P.
If we see Fig. 14, two predicted damage days, ̂ can be found in the row numbers of 4 (predicted damage day(1)) and 6 (predicted damage day (2)) as ̂ =1. RR is 2 days and this is regarded as ±2 days around ̂ =1. Therefore, RR(1) and RR (2) can be considered as the risk range based on predicted damage day(1) of ̂= 1 and also RR(2) and RR(3) based on predicted 415 damage day(2). The random numbers are generated for the days corresponding to RR and this becomes the probability(P) that AD is occurred. However, RR(2) in predicted damage day(1) is overlapped with predicted damage day(2) and so the random number of 0.24 is produced just for one day. That is, P is generated and compared with PP for the determination of ′ . If we see Figure 15, since the row numbers of 1 and 9 are not included in RR, ′ becomes 0. The generated random numbers for the row numbers of 2, 5, 7, and 8 are less than PP=0.32 and so ̂= 0 becomes ̂′ =1. In the row 3, since P=0.84 is greater than 420 PP=0.32, ′ =0. ̂ is 1 in the rows of 4 and 6 and so ̂′ maintains 1. In this way, the analysis is performed for 5 cases and this procedure is defined as Advanced HD-TRC. That is to say, Advanced HD-TRC considers uncertainty of AD by applying MCS and obtains more improved ̂′ .

Determination of the Best Criteria by the Performance Evaluation
The HD-TRC and Advanced HD-TRC for each Case were determined in training set (2005 ~ 2014). Also, these criteria were evaluated in test set (2015 ~ 2018) then the best criteria was obtained. All the results including HRA are tabulated in tables 12 to 14. As the results, HD-TRC shows much better than HRA of KMA in its performance. If we see F1-score to compare HD-TRC with HRA, the performance of HD-TRC on Gyeonggi-province is improved 23.17%, Seoul-city 5.06%, and Incheon-430 city 19.37%. F1-scores of Advanced HD-TRCs for 5 Cases show better than HRA and HD-TRC. Advanced HD-TRC for Case 4 has the best performance in Gyeonggi-province and Incheon-city, and one for Case 5 has in Seoul-city. Therefore, the proper best criteria for warning and forecasting of heavy rain damage will be Advanced HD-TRC 4 in Gyeonggi-province and Incheon-city, and Advanced HD-TRC 5 in Seoul-city.

Summary and conclusions
We have suggested a methodology for the derivation of the rainfall criteria which heavy rain damage can occur. To do this, 440 data mining process was used and the performance of the derived rainfall criteria was evaluated systematically. Initial step includes defining Rain Index (RI) by applying principal component analysis for various rainfall variables and all RIs were considered as candidates of HD-TRC. The performance evaluation for RIs based on heavy rain damage data was carried out and the best HD-TRC was selected. However, even though we do not have rainfall or have small rainfall amount during DP, 'abnormal days'(ADs) that the damage occurs exist. Therefore this study considered HD-TRC and the occurrence probability 445 of ADs using Monte Carlo simulation. Then we defined this process as Advanced HD-TRC. Then, three criteria of HRA of KMA, HD-TRC, and Advanced HD-TRC were compared through classification performance and the main results of this study are summarized as follows: (ⅰ) The best HD-TRCs in regions showed a value of 5.030 in and 5.109 in Incheoncity. 450 (ⅱ) The performance of HD-TRC by F1-score has improved compared to the HRA by 23.17% in Gyeonggi-province, 5.06% in Seoul-city, and 19.37% in Incheon-city.
(ⅲ) The performance of the 5 Advanced HD-TRCs by F1-score has improved compared to the performance of HD-TRC for each region.
(ⅳ) The best criteria for each region were selected as Advanced HD-TRC 4 in both 455 and Advanced HD-TRC 5 in Seoul-city.
(ⅴ) The performance of the best criteria for each region was evaluated by F1-score and AUC. The results are as follows: F1-score showed 37.94% in Gyeonggi-province, 30.04% in Seoul-city, 40% in Incheon-city.
AUC showed 73.36% in  Upon calculating the F1-scores, HD-TRC showed an improvement of 15% on average compared to HRA of KMA in all regions, while Advanced HD-TRC showed a 21% improvement. In addition, if we see evaluation results by AUC, HD-TRC was improved by 7.8% compared to HRA, while the Advanced HD-TRC improved by 20%. F1-score, which is known to be suitable for imbalanced data, was used as a performance measure. However, F1-score is a relative evaluation indicator, that is, it depends on the degree of imbalance in the data. Therefore, AUC, which is an absolute evaluation indicator was also used in 465 this study.
It was identified that the HD-TRC of Seoul City is the highest and the HD-TRC of Gyeonggi Province is the lowest. This signifies that the region with a high HD-TRC means that the damage can occur on relatively high rainfall intensity. It could be judged that roughly, the CDP of Seoul City is higher than that of Incheon City and Gyeonggi Province. In this study, since only the rainfall and damage data were used, the relative CDP could be roughly estimated. However if additional factors such 470 as socio-economic factors or disaster prevention projects in future research are to be considered, it could be expected that the CDP in region can be explicitly identified. If the results of this study would be used for an early warning system, it is possible to support the disaster managers through predicting the occurrence of heavy rain damage. Thus, it is expected that the suggested methodology will contribute in establishing safe environment of our life from natural disaster such as heavy rainfall.