A Web-based Crowdsourcing Tool for Manually Verifying Flood Photos and Extracting Water Height Point
In our flood mapping model, we define Water Height Point (WHP) as an estimation of a flooding situation (water level) at a specific location and time during a flooding event. A WHP can be considered as an event containing at least three elements: geographic coordinates of a location, estimated water height at this location, and the time associated with the water height. WHPs can be derived from different data sources such as a georeferenced tweet message describing the flood depth, a photo showing the flood, or stream gauge measurement of water height. In this study, we extracted the WHPs from two sources: tweets (text and photo) and stream gauges.
In our studies (Li et al., 2017 and Huang et al., 2018), georeferenced flood-related tweets within the study area were manually checked to select the tweets that indicate flooding. Water height was then estimated for each selected tweet based on the text description and/or visually examination of the flood-related photos. For example, from “10/5/2015 12:09: Flood water is still over 18 inches deep on Wildlife Road near Industrial Road”, a WHP with the water height of 457 mm is created for this location.
An online collaborative tool was developed to facilitate this verification and water-height estimation process. This tool displays flood-related tweets on Google maps and allows users to intuitively compare the text/photo against different types of maps such as road map, satellite imagery, and street views.
Automatically extracting flooding photos from social media with deep learning
This research aimed to classify disaster photographs from social media and we used the flooding photograph detection as a case study. We downloaded 39,000 photographs from 140,000 tweets and classified them into flooding and non-flooding photographs to build a training dataset. Several common architectures of neural network were trained, and total accuracies range from 90% to 93%. The study demonstrates that the image classification technique can be used to detect in-situ flooding photograph; this timely visual information may be helpful to the disaster management agency. Additionally, we summarized the standard of identifying flooding and non-flooding photographs, which could be modified to fit other types of disaster.
The following figure shows some of the photographs identified as flooding by the neural network in Houston Flood 2017 dataset.
Automatically identify disaster-related social media posts(tweets) with deep learning
Social media platforms renders us an important way to share information online, especially during severe disaster events. Social media including Twitter, Facebook and Flickr empower millions of private citizens, as eyewitnesses, to voluntarily document their observations and thoughts in a highly up-to-date manner. Given the fact that on-topic social media only comprises a small proportion of the enormous volume of information in social media space, the practicality of social media has been greatly hampered by the limited approaches of automatic on-topic social media retrieval.
This research presents an automatic approach to labelling on-topic social media posts using a visual-textual fused vector. The fused feature proves that additional visual feature leads to more robustness compared with the situation where only textual feature is used. The on-topic posts, classified by their texts and pictures in an automatic manner, represent timely disaster documentation during an event. Coupling with rich spatial contexts when geotagged, social media could greatly aid in a variety of disaster mitigation approaches.
Li Z., Wang C., Emrich C., Guo D. (2018) A novel approach to leveraging social media for rapid flood mapping: a case study of the 2015 South Carolina floods, Cartography and Geographic Information Science, 45(2), https://doi.org/10.1080/15230406.2016.1271356
Huang X., Li Z., Wang C., Ning H., (2020), Identifying disaster related social media for rapid response: a visual-textual fused CNN architecture, International Journal of Digital Earth, 13(9), https://doi.org/10.1080/17538947.2019.1633425
Huang X., Wang C. Li Z. Ning H.,(2019) A visual-textual fused approach to automated tagging of flood-related tweets during a flood event, International Journal of Digital Earth, 12(11), https://doi.org/10.1080/17538947.2018.1523956
Ning H., Li Z., Hodgson M., Wang C., (2020), Prototyping a Social Media Flooding Photo Screening System Based on Deep Learning, ISPRS International Journal of Geo-Information, 9(2), 104; https://doi.org/10.3390/ijgi9020104
Ning H. (2019). Prototyping A Social Media Flooding Photo Screening System Based On Deep Learning and Crowdsourcing (Committee: Drs. Zhenlong Li, Michael E. Hodgson, Cuizhen Wang)