Recently, deep convolutional neural networks (CNNs) have been used for ear recognition with the increasing and available ear image databases. However, most known ear recognition methods may be affected by selecting and weighting features; this is always a challenging issue in ear recognition and other pattern recognition applications. Metric learning can address this issue by using an accurate and efficient metric distance called Mahalanobis distance. Therefore, this paper presents a novel approach for ear recognition problems based on a learning Mahalanobis distance metric on deep CNN features. In detail, firstly, various deep features are extracted by adopting VGG and ResNet pre-trained models. Secondly, the discriminant correlation analysis is exploited to eliminate the dimensionality problem. Thirdly, the Mahalanobis distance is learned based on LogDet divergence metric learning. Finally, K-nearest neighbor is used for ear recognition. The experiments are performed on four public ear databases: AWE, USTB II, AMI, and WPUT, and experimental results prove that the proposed approach outperforms the existing state-of-the-art ear recognition methods.