This study assesses the performance of three classification trees (CT) models (Entropy, Gain Ratio and Gini) for building detection by the fusion of airborne laser scanner data and multi-spectral aerial images. Data from four study areas with different sensors and scene characteristics were used to assess the performance of the models. The process of performance evaluation is based on four criteria: model validation and testing; classification accuracies; relative importance of input variables; as well as transferability of classification trees derived from one dataset to another. The lidar point clouds were filtered to generate a Digital Terrain Model (DTM) based on the orthogonal polynomials, and then a Digital Surface Model (DSM) and the Normalized Digital Surface Model (nDSM) were generated. A set of 26 uncorrelated feature attributes were derived from the original aerial images, lidar intensity image, DSM and nDSM. Finally, the three classification trees models were used to classify buildings, trees, roads and ground from aerial images, lidar data and the generated attributes with the most accurate average classifications of 95% being achieved. The Entropy splitting algorithm proved to be a preferable algorithm for building detection from aerial images and lidar data.