Urban sensing and understanding using big visual data

Cites are being watched by an increasing number of cameras. Besides the conventional traffic and security cameras, others are found in smartphones, self-driving vehicles, and drones. Massive visual data are being collected every day around the world and the volume keeps growing. For example, Instagram users upload millions of photos per hour; Google Street View provides images for most streets in major cities worldwide; autonomous cars gather images around them every second when running on the roads.

These big visual data, combined with embedded location information, offer unprecedented opportunities to discover patterns and knowledge in urban environments. For example, analyzing massive images/videos captured in urban areas can help researchers uncover urban phenomena quantitatively and qualitatively, such as how visitors use public parks, what kind of people visit a landmark frequently, or where is being gentrified. Besides resident behavior analysis, municipal facility administration also benefits from harnessing urban images/video, for example, street furniture inventorying, sidewalk mapping, street tree species detection and diameter measuring, and neighborhood walkability assessments.

The tremendous advancements in artificial intelligence and computer vision over the last decade have resulted in powerful tools for extracting semantic information from images/videos. However, it is unclear that what kind of new technology and data sources can be used or need to be developed, and how they help people to capture the dynamic of urban life and to understand the interaction between residents and urban environments.

Our studies aim to use big visual data to sense and understand urban environments, including conceptualization, knowledge framework, toolbox organization, and applications. The ultimate goal is to enhance the productivity of urban management and the life of city residents.

We project a street view image into a landcover map, combining the GeoScience and the deep learning technique.


Ning H., Li Z., Wang C., Hodgson M., Huang X., Li X., (2022) Converting street view images to land cover maps for metric mapping: a case study on sidewalk network extraction for the wheelchair usersComputers, Environment and Urban Systems.  https://doi.org/10.1016/j.compenvurbsys.2022.101808

Ning H., Li Z., Ye X., Wang S., Wang W., Huang X., (2022). Exploring the vertical dimension of street view image based on deep learning: a case study on lowest floor elevation estimationInternational Journal of Geographical Information Science, 36(7). 1317-1342. https://doi.org/10.1080/13658816.2021.1981334

Li, X., Ning, H., Huang, X., Dadashova, B., Kang, Y., & Ma, A. (2022). Urban infrastructure audit: an effective protocol to digitize signalized intersections by mining street view imagesCartography and Geographic Information Science, 49(1), 32-49. https://doi.org/10.1080/15230406.2021.1992299

Song, Y., Ning, H., Ye, X., Chandana, D., & Wang, S. (2022). Analyze the usage of urban greenways through social media images and computer visionEnvironment and Planning B: Urban Analytics and City Science, 23998083211064624. https://doi.org/10.1177/23998083211064624

Ning H., Li Z., Hodgson M., Wang C., (2020), Prototyping a Social Media Flooding Photo Screening System Based on Deep Learning, ISPRS International Journal of Geo-Information, 9(2), 104; https://doi.org/10.3390/ijgi9020104

Ning H., Huang X., Li Z., Wang C., Xiang D., (2020) Detecting New Building Construction in Urban Areas Based on Images of Small Unmanned Aerial SystemPapers in Applied Geography, https://doi.org/10.1080/23754931.2019.1707108

Huang X., Wang C. Li Z. Ning H.,(2019) A visual-textual fused approach to automated tagging of flood-related tweets during a flood eventInternational Journal of Digital Earth, 12(11), https://doi.org/10.1080/17538947.2018.1523956

Translate »