The existing pedestrian detection algorithms are not robust in the case of noise, obstruction and illumination change. To solve the problem, we propose a pedestrian detection algorithm combining the edge features of color image with the features of Histogram of Oriented Gradient on Depth image (referred to as the HOD features). The algorithm describes overall structural features of pedestrians by using shearlet transform to extract their edge features from color images, and obtains local edge features of corresponding depth images by generating HOD features. The overall structural features and local edge features are combined to form new feature descriptors to train a SVM (support vector machine) classifier. Due to the full integration of the two types of features, the algorithm shows significant advantages in pedestrian detection in the case of interfering factors such as noise, obstruction, illumination, and similar colors. The experimental results show that the detection accuracy rate of this algorithm is 15% higher than that of other algorithms when the false-positive rate is 0.1.