How to build a training dataset from a large landcover map inventory?

Land cover data is an inventory of objects on the Earth’s surface, which is often derived from remotely sensed imagery. Deep Convolutional Neural Network (DCNN) is a competitive method in image semantic segmentation. Some scholars argue that the inadequacy of training set is an obstacle when applying DCNNs in remote sensing image segmentation. While existing land cover data can be converted to large training sets, the size of training data set needs to be carefully considered. In this paper, we used different portions of a high-resolution land cover map to produce different sizes of training sets to train DCNNs (SegNet and U-Net) and then quantitatively
evaluated the impact of training set size on the performance of the trained DCNN.

We also introduced a new metric, Edge-ratio, to assess the performance of DCNN in maintaining the boundary of land cover objects. Based on the experiments, we document the relationship between
the segmentation accuracy and the size of the training set, as well as the nonstationary accuracies among different land cover types. The findings of this paper can be used to effectively tailor the existing land cover data to training sets, and thus accelerate the assessment and employment of deep learning techniques for high-resolution land cover map extraction.

The following is our findings:

In the rural area, where less infrastructure exists, DCNN needs fewer data to learn the features and can get better performance on class detection and boundary depiction.
In an imbalanced training set, classes with limited coverage of the study area often have poor accuracies. Data augmentation and training tuning may be needed to improve the performance of DCNN.
Deep learning techniques are not suitable for certain applications in land cover segmentation. Some land cover classes in this research, such as Structure, Excavated land, Grass, and Orchard, trend to be misclassified as other classes with similar textures in 0.5 m RS images.
The new Edge-ratio metrics can be used to evaluate the segmentation results. The result with an Edge-ratio closer to the ground truth has a better representation of the class boundaries.

Fig. 1 Experimenta dataset. A landcover map inventory was divided into 140 tiles, and different numbers of tiles were selected to form size-variant training datasets.

Fig. 2 Segmentation results from SegNet and U-Net. The training set of 120 tiles achieve the best results.

Fig. 3 Urban areas benefit from large training sets.

Fig. 4 Rural areas benefit less from large training sets.

Fig. 5 Aurracy changes among categories. Pay attention to your target object!

Fig. 6 Examples of edge-ratio. Smother landcover maps have smaller edge-ratio.

Fig. 7 Edge-ratios of results from training sets. U-Net generated smother boundaries, but still have a large gap to the ground truth, indicating further improvements.

Publication: Ning, Huan, Zhenlong Li, Cuizhen Wang, and Lina Yang. “Choosing an appropriate training set size when using existing data to train neural networks for land cover segmentation.” Annals of GIS 26, no. 4 (2020): 329-342. https://doi.org/10.1080/19475683.2020.1803402