Deep learning (DL)-based categorization and detection methods for breast cancer diagnosis through medical images are utilized by computer-aided diagnosis (CAD) techniques. In this study, two DL-based ensemble models were proposed to improve the diagnostic efficiency of the system. The proposed models were evaluated on two mammography datasets: DDSM and CBIS-DDSM. To enhance the performance of breast lesion classification from mammographic scans, three pretrained convolutional neural network (CNN) models, namely VGG16, InceptionV3, and VGG19, were used as base classifiers, and two ensemble models were trained. Ensemble Model 1, using a linear meta-learner in the form of logistic regression for classification, and Ensemble Model 2, using a neural net as the meta-learner for classification. The DDSM dataset achieved accuracy, sensitivity, and specificity of 98.02%, 97.17%, and 98.87%, respectively, for Ensemble Model 1, and 98.10%, 97.01%, and 99.12%, respectively, for Ensemble Model 2. Furthermore, the performance of the proposed models was compared with existing state-of-the-art systems, and the results showed an increase in breast cancer classification performance with the proposed models. Hence, the proposed models have the potential to aid medical professionals in accurately classifying breast lesions.