Dear Vincent G. Schippers and Wouter,
Thank you for the submission of your manuscript “Uncovering the veil of night light changes in times of catastrophe”. As you know, three reviewers have now provided detailed reviews, which you have replied in thoughtful detail to. Two reviewers are quite critical and recommend major revisions, while one reviewer suggests only minor revisions.
As I consider the concerns about the suitability and validity of the method you have used to be very serious and substantial, I invite you to undertake a major revision of your manuscript. As said, major concerns consider the method, the other important point is to improve the structure and text of your manuscript. Doubling of messages and wordy, lengthy descriptions need to be avoided. Please try to be as concise and clear as possible.
Please also provide an ‘author’s reply’ to the reviewers (feel free to use (in parts) the same words that you used in what you have already uploaded). Please can you also include a track changes document between the old manuscript and the new one (you can include this as part of your ‘author’s reply’).
I look forward to seeing the next version of your manuscript which I will then send out for further review to either the previous reviewers (if they agree) or new reviewers.
Best Regards
Heidi Kreibich
NHESS Executive Editor

Dear Vincent G. Schippers and Wouter J. W. Botzen,

Two reviewers have now provided reviews of your revised manuscript, with very different recommendations. Reviewer 1 is happy about the revision you have undertaken and suggests accepting your manuscript for publication. Reviewer 2 suggests to reject your manuscript, sine you have not dealt with any of the major concerns that were identified during his first review. Additionally, none of his suggestions were followed.

Therefore, I would like to invite you to submit a revised version of your manuscript, considering more carefully the major problems identified and suggestions for improvement provided. I understand, that it is not in scope of the paper to extent the analyses from 8 counties to the whole of the US and from one hurricane event to multiple hazard events. On the other hand, I agree with the reviewer that the following aspects need to be solved and proven: 1) It is important to demonstrate that your approach to using night light data actually measures what you claim to measure. The smaller the case study and the more specific (only one event), the more you can tweak the approach to get the results you want (overfitting), with the risk that the approach will not give reasonable results elsewhere (other region/other events). 2) You need to demonstrate that your approach does not involve significant bias. 3) The claim that the approach is generally applicable (perhaps even "in low-income countries") is critical, as only one application (without a transferability test) is shown.

I copy the review from the previous round below, and hope that you can come up with convincing solutions for the mentioned problems.

I look forward to seeing the next version of your manuscript which I will send out for further review.

Best regards
Heidi Kreibich
NHESS Executive Editor


Anonymous Referee #3
This paper aims at assessing “to what extent it is possible to capture the regional economic dynamics following damages from a big natural disaster by making use of the annual nighttime lights” (p. 2) especially in countries with poor quality of administrative data. The paper addresses this question by using hurricane Katrina as a test, which hit the US, a country with comparative high-quality public administrative data, in 2005. Using descriptive statistics on eight counties damaged most severely by Katrina, they find that changes of DMSP-OLS night light data are positively correlated with changes of population, employment, real GDP and household income in these counties after 2005. The authors conclude from these positive correlations that “night light changes reflect the general pattern of direct impacts of Katrina as well as the subsequent recovery” (p. 21).

Major comments
The research question of the paper is interesting and relevant in my view. Night light data may help assess the damages of natural disasters in countries or regions where reliable official data is not available.

The empirical approach adopted in the paper, the correlation analysis in Section 3.3, is not suited too well for providing a reliable answer to this question, however, for at least three reasons. The first is that, focusing on only 8 severely damaged counties, this correlation analysis may not be representative of all counties hit by Katrina, not to mention regions elsewhere hit by other hurricanes. The second reason is that this correlation analysis lacks a benchmark, i.e., does not control effectively for the regular relationship between night lights and economic indicators in “normal” times. The paper presents correlations for the 8 counties in the five pre-Katrina years but this analysis produces weird results (negative rather than positive correlations). Taken at face value, these correlations suggest that night lights are poor predictors of economic activity in the 8 counties in “normal” times. Why should they predict economic activity more reliably in times of disaster? And third, the results may be biased by top-coding of the night light data.

The authors may consider using a more sophisticated empirical approach that can be expected to yield a more reliable answer to the core question of their paper. I suggest using a panel regression that starts from the premise that the relationship between night lights and economic indicators is stable over time and across space in “normal” times, and test if this relationship is different in times of disaster (remark: If the stability assumption is violated even in normal times, the whole exercise is pointless anyway). One way to do this test, in accordance with Henderson et al. (2012), would be estimating the model
ln(L(i,t) = a(0) + a(1)*ln(Y(i,t) + b(2005)*ln(Y(i,t)*D(i)*2005 +
b(2006)*ln(Y(i,t)*D(i)*2006 + ... + b(2012)*ln(Y(i,t)*D(i)*2012 + county fixed effects + year fixed effects + residual(i,t),
or all 3,000something US mainland counties (indexed by i) and years t = 2000, ..., 2012. L(i,t) denotes average night light intensity, A(i) area in sqkm, Y the economic indicator of interest (e.g., GDP, population, employment), D(i) a measure of the damage induced by Katrina in county i and the respective year after Katrina, and 2005...2012 year dummies. D may be a dummy variable that is set to 1 for the counties hit by Katrina (maybe above a some threshold damage), or may be defined as ln(1 + % of damaged housing). The parameter a(1) reflects the elasticity of night lights w.r.t. the Y in “normal” times, identified from all annual observations in the counties unaffected by Katrina and the pre-Katrina annual observations in the counties hit by Katrina. The parameters b(2005) ... b(2012) of the year-specific interactions between the economic variable of interest and disaster will reflect annual deviations from these normal elasticities in times of disaster, identified from the post-Katrina annual observations in all counties hit by Katrina. The null hypothesis is that these parameters are zero, in which case night lights can – conditional on the stability of a(1) - be considered a reliable predictor of the respective Y in times of disaster because the elasticity is unaffected by the disaster. Setting D=ln(1 + % of damaged housing) even allows for testing if this elasticity is invariant to the intensity of the damages induced by Katrina. Robustness checks should address the biases due to top coding, as in Henderson et al. (2012).

I find the lengthy verbal descriptions of the association between changes of night light intensities and economic indicators in Sections 3.1 and 3.2 rather uninformative and confusing. I suggest skipping them. In addition to this, Section 4 may be dropped after moving the few points not made elsewhere in the paper to other sections.

Minor comments
I suggest concentrating discussion of the three methodological issues of the night light data (intertemporal differences, top coding, overglow) in a single subsection. The discussion of overglow on p. 9 (FN 10) is misplaced in my view.
I strongly suggest using average night light intensities (by square kilometer) rather than sums of night light intensities across pixels throughout the paper. The sums do not control for differences in the geographic sizes of the counties. This is particularly relevant for Figure 3.
I also suggest using a single measure of changes of night intensities over time, percentage changes, consistently throughout the paper. Currently, the paper discusses absolute changes in some Figures (3, 7) and percentage changes (or indexes) in others.
I suggest either harmonizing the spatial scales of the maps in Figure 2, or putting the upper left map into a separate figure.

Dear Vincent G. Schippers and Wouter J. W. Botzen,

I am pleased to inform you that the revised version of your manuscript “nhess-2022-23 Uncovering the veil of night light changes in times of catastrophe” can now be accepted for publication in our journal Natural Hazards and Earth System Sciences (NHESS).

I take this opportunity to thank you for having selected Natural Hazards and Earth System Sciences (NHESS) for the publication of your scientific work. I look forward to receiving further contributions to our journal from you and your colleagues.

Best regards

Heidi Kreibich
NHESS Executive Editor