Nowadays, breast cancer considered a main cause of death for women all over the world. It is defined as a group of cells that grow rapidly and causes the formation of a lump in breast tissue which leads to tumor formation which can be categorized either malignant (cancerous) or benign (non-cancerous). On the other side, mammography as a screening and diagnostic tool suffers from some limitations, especially with young women who have dense breasts. Therefore, there was a need to develop more effective tools. Thermography is an imaging tool used to record the thermal pattern. The main contribution of this paper is proposing a unique method for classifying the breast thermography images into one of three classes: normal, benign, or malignant. Additionally, bio-inspired algorithms namely, ant colony optimization (ACO) and particle swarm optimization (PSO) are used for feature selection. The proposed method contains four phases: Image preprocessing, feature extraction, feature selection, and classification. The proposed method is assessed using a benchmark thermography dataset. The experimental results show that our method has a promising performance.