Driver fatigue and losing wariness during long driving hours is considered as one of the main road accidents causes. It affects road safety directly. Road safety is a major disquieting problem, since traffic accidents endanger divers, travelers, and everyone in their scope, in addition to the road and vehicle damages. The EEG signal becomes one of the most dependable biological signals utilized to estimate the drivers' drowsiness state, although a multichannel acquiring system must be used to transmit the EEG signal. Wearing a multi-channel headset is not readily accepted by drivers. Many attempts have been done by researchers to reduce number of EEG channels used to detect drivers’ fatigue. The present study proposed utilizing only one of EEG channels signal to estimate driver fatigue state to raise the acceptance of the system and its flexibility. The system starts with receiving the EEG signals, then pre-processing them using filtering and transformed them to color image using spectrogram. After that, the EEGs spectrogram passed to the proposed CNN deep network model to identify them either fatigue or normal fatigue. The present study measured up many EEG channels to identify the most accurate and dependable one to classify driver fatigue. The results indicate that the FP1, T3, and Oz channels considered as the most efficient channels to identify the drive’s state either fatigue or not. They achieved an accuracy of 94.33%, 92.57 and 93% respectively. Therefore, using a single one of these channels and the proposed CNN model will lead to a more robust driver drowsiness/fatigue detection system using EEG signals.