Semi-supervised Elastic net for pedestrian counting

Abstract

Pedestrian counting plays an important role in public safety and intelligent transportation. Most pedestrian counting algorithms based on supervised learning require much labeling work and rarely exploit the topological information of unlabelled data in a video. In this paper, we propose a Semi-Supervised Elastic Net (SSEN) regression method by utilizing sequential information between unlabelled samples and their temporally neighboring samples as a regularization term. Compared with a state-of-the-art algorithm, extensive experiments indicate that our algorithm can not only select sparse representative features from the original feature space without losing their interpretability, but also attain superior prediction performance with only very few labelled frames.

Introduction

Pedestrian counting in public places plays a key role in many applications, such as evacuating from a dense region to a sparse one when an emergency happens [1], or optimizing the design of traffic infrastructures to provide better transportation services. Furthermore, social security and surveillance strongly depend on the effectiveness of pedestrian counting [2].

Generally speaking, estimating the number of pedestrians can be performed in two different manners. The first approach is based on detection or tracking methods, e.g. Leibe et al. [3] and Kim and Cipolla [4]. Such methods could count pedestrian number accurately when pedestrian density is low. As the density increases, however, the performance of these methods will deteriorate since occlusion will become obvious in the dense crowd. Other aspects such as variances in pedestrian including height, pose and large bags will also impair the performance. The second approach directly estimates the number of pedestrians in a scene through using pedestrians’ pixel and texture information. Without identifying individuals, one remarkable advantage of this approach is that pedestrians’ privacy can be well preserved [5]. Assuming that the number of pixels occupied by pedestrians has a linear relationship with pedestrian counts, Davies et al. [6] used the pixel information to count pedestrians. However, such a way suffers from (1) perspective effect, which means that human size in an image changes as the distance from the subject to the camera varies, and (2) overlapping between pedestrians. For example, the number of pixels occupied by two overlapped pedestrians is less than that of pixels occupied by two non-overlapped pedestrians if the distances between pedestrians and camera keep the same. Therefore, it invalidates the assumption of the linear relationship mentioned above.

To solve the perspective issue, Chan et al. [5] proposed a perspective normalization map in which each entry in the map is regarded as a normalization factor on the corresponding pixels, giving larger factors to distant pixels in which the corresponding detected areas are far away from camera. Ma et al. [7] proposed another perspective correction matrix, based on the fact that a person in the scene is vertical to the ground plane, whether standing or walking. As for the overlapping issue, one way for this is to directly estimate the pedestrian density from each image. Marana et al. [8] claimed that some textures, such as Gray Level Co-occurrence Matrix (GLCM), can describe the crowd density well since the dense crowd always has finer texture, whereas sparse crowd is of coarser texture. Observing that pedestrian edge is also an effective feature to describe the pedestrian density, Marana et al. [9] proposed Minkowski fractal dimension and Kong et al. [10] employed edge orientation histogram based on the pedestrian edge. Chan et al. [5] further refined the performance with the combination of Minkowski fractal dimension and edge orientation histogram features.

Note that all the aforementioned pedestrian counting algorithms are achieved under the supervised learning framework. A major disadvantage for these algorithms is that, to obtain a good performance, it is necessary to collect a great number of labelled frames in the learning process. However, it is a boring and labor-intensive work to label too many frames. In addition, mislabeling often happens easily during manual labeling.

Fortunately, there are a large number of unlabelled frames which provide useful topological information that can benefit to the performance of pedestrian counting. In the machine learning community, semi-supervised learning framework, which utilizes unlabelled samples effectively, has been extensively investigated in last decade [11]. One representative strategy of semi-supervised learning is to use the regularization technique to utilize structure of unlabelled data. Zhu [11] proved that if the unlabelled data structure can approximate the true population distribution well, an expected result will be obtained. To better exploit information in unlabelled data, Zhu and Goldberg [12], Zhang et al. [13] and Hou et al. [14] introduced domain knowledge to build the regularization term for regression and dimension reduction, respectively. It is worth noting that compared with semi-supervised classification, semi-supervised regression is less studied. The main reason is that in classification tasks, the number of categories is finite, whereas object values are continuous in regression tasks. Zhou and Li [15] proposed a co-training style algorithm in which two different regressors make their respective estimations on unlabelled data, and refine the performance by learning the results of each other. Rwebangira and Lafferty [16] proposed a semi-supervised locally linear regression through adding a weighted regularization term to the regression function. However, it is not easy to directly generalize these semi-supervised approach to pedestrian counting since (1) the difficulty of seeking two proper views or distant metrics on pedestrian data for co-training, and (2) the local linear property does not always be true for pedestrian data with rich features.

Another common issue in pedestrian counting is that there are many redundant features to be removed when a collection of features are extracted from pedestrian video. Although Lasso proposed by Tibshirani [17] can be helpful to reduce such features, its efficiency depends greatly on the number of dimensions. When the pairwise correlation between a group of features is high, lasso chooses only one feature from the group. To overcome these two limitations, Elastic net, a variant of Lasso [17], is proposed by adding a L₂ norm constraint [18]. Therefore, we attempt to utilize the Elastic net to reduce the redundant features which have less relationship with the properties of pedestrians.

To better reflect the degree of the pedestrian density and effectively utilize the structural information in the unlabelled pedestrian image sequence, we also introduce statistical landscape features (SLF) proposed by Xu and Chen [19] to extract statistical features closely related to the property of pedestrian counts. Then we propose semi-supervised Elastic net (SSEN) by incorporating sequential correlation between frames into the Elastic net. With this way, we can remove redundant features from SLF, and meanwhile utilize sequential information in the unlabelled frames to improve the performance of counting pedestrians.

The main contributions of the proposed SSEN algorithm include: (1) To the best of our knowledge, this is the first time to employ semi-supervised learning to solve the pedestrian counting problem. (2) Domain knowledge which refers to sequential information extracted from pedestrian videos is elegantly utilized. (3) Without losing interpretability, the original features can be effectively reduced by the proposed SSEN algorithm. (4) Compared with the state-of-the-art pedestrian counting algorithm [5], which generally requires hundreds of labelled frames to obtain a relatively good result, SSEN achieves better performance, even with only very few labelled frames.

The remainder of this paper is organized as follows. In Section 2, we describe the procedure of feature extraction, and in Section 3, we detail our SSEN algorithm. Experimental results are provided and analyzed in Section 4. Finally, Section 5 concludes this work.