Abstract: One of the biggest risk factors among women is breast cancer which leads to death. Times over the past decades, the overall survival rate in breast cancer has improved due to advancements in early-stage diagnosis and tailored therapy. Women's deaths could be minimized if breast cancer is discovered early. Though traditional breast cancer treatment takes so long, early cancer techniques require an automation system. This research provides a new methodology for classifying breast cancer using ultrasound pictures that use deep learning and the combination of the best characteristics. Initially, after successful learning of Convolutional Neural Network (CNN) algorithms, data augmentation is used to enhance the representation of the feature dataset. Then it uses BreastNet18 with fine-tuned VGG-16 model for pre-training the augmented dataset. For feature classification, Entropy controlled Whale Optimization Algorithm (EWOA) is used. The features that have been optimized are EWOA was utilized to fuse and optimize the data. To identify the breast cancer pictures, training classifiers are used. By using the novel probability-based serial technique, the best-chosen characteristics are fused and categorized by machine learning techniques. The testing was performed using a dataset of enhanced Breast Ultrasound Images (BUSI). The proposed method improves the accuracy compared with the existing methods.