Segment-based Classification of Mobile Lidar Point Clouds with Limited Samples

Abstract

Mobile lidar data have been widely used in building 3D models, road mapping and inventorying, and nowadays in driverless car technology. Compared with traditional photogrammetry and remote sensing data acquisition methods, mobile lidar technology can collect precise 3D point cloud data more efficiently at driving speeds in urban environments, even in extreme weather conditions. Efficient processing of mobile lidar data, including object detection and recognition, is an active research field. Manual object detection and labelling is tedious, and it is limited by the variety of objects and the complexity of the environments in which the data is acquired. Therefore, the development of automated and efficient object recognition methods is important, but also challenging.

A common procedure for automatic processing of 3D lidar data includes successive segmentation, classification and labeling of objects from initially unstructured point clouds. This segment-based classification of mobile lidar data essentially relies on local and global features extracted from point coordinates. These features are either hand-designed, as in traditional supervised machine learning, or automatically learned by more recent deep-learning-based methods.

Compared with the traditional supervised machine learning methods, deep convolutional neural networks can learn high-level representations through compositions of low-level point information from large numbers of training samples. However, despite their remarkable success, deep networks require a large number of training samples which makes their application to mobile lidar point clouds very problematic. To overcome the limitation of training samples, transfer learning and domain adaptation methods have been introduced with the aim of transferring available information or knowledge from a source domain to a different target domain. The transfer learning methods can be roughly divided into two categories: shallow and deep. The shallow transfer learning methods such as weighting-based, feature-align-based and model-adjust-based have gained popularity for their succinctness and operability at the cost of shallow transferability. In contrast, end-to-end deep transfer learning methods have better high-level common feature extraction ability and better transferability.

The aim of this research is to develop and evaluate methods for accurate segment-based classification of mobile lidar point clouds with limited training samples. The main contributions of this research are as follows. First, the ability of traditional machine learning using local feature extraction and encoding methods in classification with limited samples is investigated. Second, a method is proposed to take advantage of available complementary datasets by combining feature extraction in a deep network and shallow transfer learning by sample reweighting. Third, an end-to-end deep transfer learning method is proposed by extending a domain adaptation network from 2D to 3D for application to mobile lidar point clouds. Experimental evaluation of the methods indicates the significant potential of transfer learning methods to overcome the limitation of training samples and improve the classification accuracy of mobile lidar point clouds.