Strong hurricane winds damage power grids and cause cascading power failures. Statistical and machine learning models have been proposed to predict the extent of power disruptions due to hurricanes.
Existing outage models use inputs including power system information, environmental parameters, and demographic parameters. This paper reviews the existing power outage models, highlighting their strengths and limitations. Existing models were developed and validated with data from a few utility companies and regions, limiting the extent of their applicability
across geographies and hurricane events.
Instead, we train and validate these existing outage models using power outages from multiple regions and hurricanes, including hurricanes Harvey (2017), Michael (2018), and Isaias (2020), in 1910 US cities. The dataset includes outages from 39 utility companies in Texas, 5 in Florida, 5 in New Jersey, and 11 in New York. We discuss the limited ability of state-of-the-art machine learning models to (1) make bounded outage predictions, (2) extrapolate predictions to high winds, and (3) account for physics-informed outage uncertainties at low and high winds.
For example, we observe that existing models can predict outages higher than the number of customers (in 19.8 % of cities with an average overprediction ratio of 5.2) and cannot capture well the outage variance for high winds, especially above 70 m s

Hurricanes can cause significant damage to the power distribution systems, resulting in large power failures and losses of billions of US dollars

Hurricane-induced power interruptions can cause billions of dollars in losses and long-lasting impacts on vulnerable communities. The power outages caused by storms can last for several hours to weeks and even months (

Utilities must first assess the vulnerabilities in their power system infrastructure to enhance their resilience to hurricanes.
Researchers have developed machine learning models to
help utilities evaluate their vulnerabilities to predicting the extent of power
outages from hurricanes. These outage models use inputs including hurricane winds, power systems, environmental information, and demographic information. Outage prediction models can assist utilities in planning and placing their resources before and during an extreme event for an emergency response to rapidly recover the failed power distribution systems

Previously, researchers have used more complex power outage prediction models, namely neural networks, kernel methods such as support vector machines, and other tree-ensemble methods, such as AdaBoost, which can model non-linear relationships between input parameters and outages

GLM

In this paper, Sect. 2 describes the input features, data sources, and data preprocessing used in the model development for power outage prediction. Section 3 explains the selection of important and uncorrelated input features for model development. GLM, GAM, and random forest power outage models are described in Sects. 4, 5, and 6, respectively. Section 7 describes the results for calibrated models and compares performance with the previous models in the literature. Section 8 highlights the limitations of existing state-of-the-art power outage prediction models to (1) make bounded outage predictions, (2) extrapolate for high winds, and (3) account for physics-informed uncertainties at low and high winds. Section 9 summarizes the findings of this paper.

We acquired power outage data from PowerOutage (

Previously,

We calibrated outage models at the city level resolution, comparable to the most recent models by

We focused on two response variables: the number of outages which are equivalent to the number of customers without power in a city and the fraction of customers without power. GLM and GAM use Poisson and negative binomial distributions to assess the count of outages as they model discrete and non-negative variables. Random forests can model the fraction of households without power in a city, which is important to compare impact levels across cities.

Power outages in aftermaths of Hurricane Isaias (2021) in New Jersey at the city level. © OpenStreetMap.

The hurricane parameters considered for this study are 3 s gust wind speed and duration of strong winds over 20 m s

Distribution of 3 s wind gust at the city level (mean: 36.95 m s

Power grid patterns vary for different land use classes, resulting in different outage mechanisms. For example, rural areas can suffer larger power outages since they have radial grid patterns where component failures can propagate more than in cities with gridded patterns

Precipitation and soil moisture have been extensively used in power outage models, e.g.,

Precipitation and soil moisture data are available from the variable infiltration capacity (VIC) model from the National Land Data Assimilation System Phase 2 (NLDAS2)

Soil moisture from NLDAS2 is available for three depths: 0–10, 10–40, and 40–100 cm. We calculated daily soil moisture for these depths by taking the average hourly readings. Soil moisture can vary at different geographical locations due to different soil types in different regions. We first normalized soil moisture to compute deviations from average values by computing percentiles.
We fit Pearson type III distributions to the daily time series of soil moisture for all three layers to normalize the soil moisture across different geographies. We use maximum likelihood estimates (MLEs) to compute the parameters for Pearson type III distribution

Precipitation data are represented in the form of the standard precipitation index (SPI)

We also included the expected precipitation after the hurricane makes landfall for the next 7 d, as heavy rain can lead to flooding resulting in clustered outages

Distribution of SPI 1 month (mean: 0.88, standard deviation: 0.93) across cities in New Jersey before the arrival of Hurricane Isaias. © OpenStreetMap.

The effective root zone depth is defined as the depth of the soil from which plants and trees can effectively extract water and nutrients for growth (

USDA created National Insect and Disaster Risk Maps

Distribution of percent treed area (mean: 73.45 %, standard deviation: 23.04 %) across cities in New Jersey. © OpenStreetMap.

Previously, researchers have found that hurricane wind speeds (and thus damages) vary with surface topography

Demographics data are available from the American Community Survey (ACS) (

Parameters to build the power outage prediction models: all variables are rescaled at the city level. Parameters are grouped into categories separated by horizontal lines. We selected one variable from each category of each group to minimize correlation across parameters.

Machine learning models with high-dimensional input data can be hard to train, especially when datasets are sparse, as in the case of infrastructure failures.
Input features can be correlated, leading to higher generalization errors.
This means the machine learning model can fit well the training data, i.e., with small errors. However, we might observe significant errors after testing the model with additional data.
Also, correlated features can lead to a flawed understanding of the relation between input and predicted outages

Feature selection, also called variable selection, is an essential step in machine learning model development to select relevant variables and discard redundant and highly correlated ones

Forward selection: selection of important input parameters based on importance to explain the variability in outage predictions. Feature descriptions are shown in Table

In the second stage, we analyzed the correlations between the input parameters. Figure S4 shows the correlation coefficients for each pair of variables.
We found that input features within the same category in Table

3 s gust wind speed

7 d precipitation

standard precipitation index 6 months

soil moisture first layer

population density

percent treed area

root zone depth.

Generalized linear models (GLMs) are a generalization of ordinary linear regression. GLMs allow us to use a flexible link function to relate a linear model (of the input variables) to the response variable

In addition, GLMs can utilize multiple statistical models to represent the data instead of the only normal distribution as in ordinary linear regressions. Outages have a lower bound of zero counts that normal distributions cannot capture. Thus, previous researchers have used the following distributions to represent outages with GLMs.

Poisson regression models, a category of GLMs, are applicable for positive count data where observations are independent. Outages are modeled as a Poisson random variable:

The variance in a Poisson distribution is equal to the mean, i.e., Var

Negative binomial GLM is a hierarchical model which can account for overdispersion effects in power outage count predictions

Researchers have also developed zero-inflation outage prediction models to improve statistical performance for unbalanced data, e.g., when there are a lot of data points with no outages

GLM models assume a linear relationship between the logarithm of the mean number of outages and input parameters (Eq. 2). However, previous research has shown that they have non-linear relationships

Random forest regressions

Example of a simple decision tree with two input features, maximum wind speed (Vmax) and precipitation (SPI6), to predict outages. The split at the root node (

Random forests “grow” a large number of parallel decision trees and bag new samples for each decision tree

Random forest models can generally capture the non-linear between the input parameters and output predictions. However, a random forest is not easily interpretable, as it is based on multiple decision trees.
In this paper, we use the sci-kit learn module in Python to fit the random forest model. We also use the GridSearchCV module in Python

In this section, we discuss the statistical performance of different outage models by first training the models on training data and then comparing the

We trained Poisson and negative binomial GLMs to predict the outage counts. The predictions are based on the seven input features mentioned in the feature selection section. All the input features are significant at a

Statistical performance measurements for generalized linear models.

The high value of residual deviance, relative to the degree of freedom, in the Poisson GLM shows large overdispersion

We also trained the Poisson and negative binomial GAMs. Like for GLMs, GAMs are trained with the seven input features mentioned before.
All the input features are significant at a

Statistical performance measurements for generalized additive models

The residual deviance for Poisson GAM (Table

The negative binomial GAM has a low value of residual deviance, which indicates that the negative binomial GAM can handle overdispersion. Additionally, non-linear shapes from spline functions (Eq.

We calibrated the random forest model to predict the fraction of customers without power.
We performed the hyperparameter tuning using the GridSearch tool in Python

Random forest outage predictions on the 20 % holdout test data.

The

We present the variable importance in the random forest model in Fig.

We found that maximum wind
speed is the most important parameter in the random forest model (Fig.

Random forest variable importance in decreasing order of importance. All importance factors are normalized by the highest value, i.e., the factor for Vmax.

Random forest and negative binomial GAMs show superior performance in predicting the power outages caused by a hurricane. MSE (mean squared error)

Different machine learning models discussed in previous sections can predict power outages for a hurricane-stricken city. Here, we discuss the limitations of state-of-the-art machine learning models for power outage predictions.

GLM and GAM regressions can predict the mean number of outages in the city. The models have a lower bound of zero as both Poisson and negative binomial distributions predict the count of outages. However, there is no upper bound on the predicted number of outages. Hence, GLM or GAM models can predict more outages than the number of customers, resulting in an overestimation of power outages.

Outage predictions on 20 % holdout test data using the negative binomial GAM outage model. Black dots represent the cities with predicted outages larger than the number of customers. Grey dots represent the cities with predicted outages less than or equal to number of customers.

For illustration, we present the power outage predictions on 20 % hold-out test data for the negative binomial GAM (Fig.

Random forest predictions are an average value of the outages in the training data (Eq.

For this assessment, we trained the random forest model on a reduced dataset, with only New York and New Jersey, and on a complete dataset, including Florida and Texas.
We present the partial dependence of wind speed in Fig.

These limited outage datasets have strong implications for the validity and extrapolability of outage models based on random forest regressions.
Under the reduced dataset, i.e., with only New Jersey and New York, outage predictions increase as the wind speed increases from

Under the complete dataset, results improve by including outages in Florida and Texas.
These states experience higher winds; e.g., their 100-year return-period winds are

In catastrophic storms, we expect large outages with higher certainty, e.g., Hurricane Ida (2021) in Louisiana

Prediction ranges of power outage as a function of wind speed for

As discussed previously,
negative binomial GAMs capture the variability of outages better than Poisson models.
Thus, we focus on the former models and estimate the variance according to Eq. (

The random forest model can only predict the mean number of outages. Thus, it cannot evaluate variances.
However, the quantile regression forest (QRF)

Moreover, structural models predict no damage to power infrastructure at wind speeds lower than 10 m s

As standard deviation is high for outages with the variation of winds, and precipitation is the second most important variable in the RF model per Fig.

Partial dependence plot of fractional outages with precipitation.

This paper summarized existing power outage prediction models, (a) GLMs and (b) GAMs based on Poisson and negative binomial distributions and (c) random forest regressions. Power outages depend on several factors, including hurricane, environmental, and demographic conditions. To examine the existing models, we used power outage data with a total of 3.6 million outages for Hurricane Isaias (2020) in New York and New Jersey states, Hurricane Harvey (2017) in Texas, and Hurricane Michael (2018) in Florida. We combined the outages from these states to develop a generalized power outage model across different regions, improving previous efforts that only calibrated outage models to a particular region or utility companies in the US. We conducted a feature selection to avoid multi-collinearity among input variables and calibrated the state-of-the-art outage models using seven input parameters: 3 s wind gust speed, 7 d precipitation after the storm, standard precipitation index for 6 months before the storm, soil moisture for a depth between 0 and 10 cm, population density, the percent area covered by trees, and trees' root zone depth.

First, we found that Poisson regressions are unsuitable for modeling outages, as historical outages have larger variances than the mean, resulting in overdispersion. The overdispersion was evident by the large residual deviances of 6 038 042 and 3 565 948 for the Poisson GLM and GAM, respectively, for 1520 degrees of freedom.
We found that negative binomial regressions account for these larger variances better than Poisson regressions since we obtained residual deviances of 2078 and 1813 for GLM and GAM, respectively.
We also showed that GAMs could better model the non-linear behavior of predictors compared to GLMs since

However, each model has its own merits and demerits in predicting outages. Poisson and negative binomial estimates are unbounded and can overestimate power outages.
For example, the negative binomial regression predicted more outages than the number of customers for

We suggest beta

The

The

RSS is the total sum of squares, defined by the sum of squares of the difference between the true value of the response variable and the predicted value of the response variable from the fitted model.

We quantify overdispersion by calculating if the residual deviance is larger than the degrees of freedom. Degrees of freedom is defined as the number of data points in the training data minus the number of input parameters.
We estimate

The value of

Power outage data are available from PowerOutage (

The supplement related to this article is available online at:

PA reviewed the existing models for power outage predictions during hurricanes, under LC's supervision. PA and LC collected and curated the data for outages and input parameters and fitted the power outage models for predicting outages. PA drafted the paper with contributions and revisions from LC.

The contact author has declared that none of the authors has any competing interests.

Publisher’s note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the special issue “Advances in machine learning for natural hazards risk assessment”. It is not associated with a conference.

We acknowledge the financial support by the NYU Tandon School of Engineering Fellowship. Additionally, this research was also supported by the Coalition for Disaster Resilient Infrastructure Fellowship Grant 210924669. The authors are grateful for their generous support.

This research has been supported by the NYU Tandon School of Engineering Fellowship. This research has been additionally supported by the Coalition for Disaster Resilient Infrastructure Fellowship (grant no. 201924669).

This paper was edited by Vitor Silva and reviewed by two anonymous referees.