Review article: Detection of actionable tweets in crisis events

Kruspe, Anna; Kersten, Jens; Klan, Friederike

doi:https://doi.org/10.5194/nhess-21-1825-2021

Articles | Volume 21, issue 6

https://doi.org/10.5194/nhess-21-1825-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Special issue:

Groundbreaking technologies, big data, and innovation for...

https://doi.org/10.5194/nhess-21-1825-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 21, issue 6

Review article

|

15 Jun 2021

Review article |

| 15 Jun 2021

Review article: Detection of actionable tweets in crisis events

Anna Kruspe, Jens Kersten, and Friederike Klan

Download

Final revised paper (published on 15 Jun 2021)
Preprint (discussion started on 10 Jul 2020)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: '577984 Review', Anonymous Referee #1, 11 Aug 2020
- AC1: 'Response to review 1', Anna Kruspe, 08 Oct 2020
RC2: 'Review of "Review article: Detection of informative tweets in crisis events"', Anonymous Referee #2, 12 Sep 2020
- AC2: 'Response to review 2', Anna Kruspe, 08 Oct 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

ED: Reconsider after major revisions (further review by editor and referees) (14 Oct 2020) by Rui Figueiredo

AR by Anna Kruspe on behalf of the Authors (18 Dec 2020) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (21 Dec 2020) by Rui Figueiredo

RR by Anonymous Referee #2 (02 Jan 2021)

RR by Anonymous Referee #3 (12 Jan 2021)

Suggestions for revision or reasons for rejection

The paper describes how information extracted from the micro-blogging platform Twitter can serve Crisis Management during a disaster rather than finding actionable tweets.
The structure of the paper is clear, and the data are exhaustively detailed. Therefore the presentation quality is good. I have significant doubts about the scientific significance and the quality of the outcomes.

Scientific significance
Contribution to the research community:
a-The vast majority of SM research in DRM uses Tweets as they are the 'de-facto' only open and free source of SM data
b-The authors start the paper in the quest for actionability but then end up describing mostly research works used for classifying relevance of tweets (4.2 and 4.3). Neither the datasets described classified tweets as 'actionable' but rather as informative or relevant.

As it is now, the paper offers an update of similar works already published (HAL, Reuter work to cite two). The novel information is the in-depth description of the models and technologies used for automated classification of tweets. The authors should leverage on the richness of the examples offered to better present the state of the art of several lines of research according to crisis management needs.

Scientific quality
My concern is that the paper's goal is not clear (or the research question not posed correctly?). I cannot understand if the article wants to offer a path for future research on finding serviceable information or describing the state of the art of SM for Crisis Management.
In the first case, the paper misses a solid base of works for supporting a choice; in the second case, there is an unclear definition for 'actionable information'.
A tentative definition is quickly described in the framing of the problem, but then it is not followed in the next chapters of the article (4.2 and 4.3). The authors describe models for classification for situational awareness in the machine learning approaches justifying it as possible pre-selection for further analysis for finding actionable information. That's perhaps confusing. There should be a more precise definition of classes of actions/decisions and clear links between each data/algorithm/technology described and such classes.

Another concern is that if the paper aims to offer a starting point for further reading, I see the risk of highlighting only some branches of the research from the community of researchers studying the intersection of Crisis Informatics and Social Media, which is significant and growing (focus seems only on the authors that published data in the datasets section/par).
Some missing ref for instance (easily found on Arxiv or Google Scholar):
Purohit H. efforts for serviceability of information is missing (Purohit, Peterson).
H2020 funded Projects such as E2MC or I-REACT are not cited while they could be listed in the crowdsourcing approach (both produced publications).
Castillo is cited but not for his last works where Convnet is used for multilingual annotation (Lorini et al., ISCRAM 19 paper award)
Again no mention of De Bruijn or Brouwer works about the detection of impact (damage assessment) very relevant together with the cited Albuquerque research about mixing authoritative and social media data.
Poblete research in Disaster Management focuses on impact detection and multilingual text.

I also think leaving out the analysis of research on Images (growing area of study) from Social Media is also a missed opportunity to describe a growing source of informative data.

Suggestion for revision:
As it is now the work has a good depth, but I am afraid is mixing different researches and experiments. It would be better, in my opinion, to change the aim of the paper as to describe how Crisis Informatics research try to extract from SM information that can be 'fit-for-purpose' in the context of Crisis Management and then divide the work into purposes (finding requests for action, situational awareness, Impact assessment, ...).
Restructuring the paper according to 'purposes' or 'needs' would simplify the 'labelling' of the described works and streamline the narrative (extending the references of course).

Hide

ED: Reconsider after major revisions (further review by editor and referees) (20 Jan 2021) by Rui Figueiredo

AR by Anna Kruspe on behalf of the Authors (05 Mar 2021) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (10 Mar 2021) by Rui Figueiredo

RR by Anonymous Referee #3 (16 Apr 2021)

ED: Publish subject to technical corrections (19 Apr 2021) by Rui Figueiredo

AR by Anna Kruspe on behalf of the Authors (23 Apr 2021) Author's response Manuscript

Short summary

Messages on social media can be an important source of information during crisis situations. This article reviews approaches for the reliable detection of informative messages in a flood of data. We demonstrate the varying goals of these approaches and present existing data sets. We then compare approaches based (1) on keyword and location filtering, (2) on crowdsourcing, and (3) on machine learning. We also point out challenges and suggest future research.