Nadia H. Alsulami, Amani T. Jamal, and Lamiaa Elrefaei, "Deep Learning-Based Approach for Arabic Visual Speech Recognition", Computers, Materials & Continua, vol: 71, No.1, pp. 85-108, 2022. doi:10.32604/cmc.2022.019450
Computers, Materials & Continua • 2022
Publication Information
Authors
Nadia H. Alsulami, Amani T. Jamal, and Lamiaa Elrefaei
Keywords
Not Available
Journal
Computers, Materials & Continua
Publisher
TSP
Volume
71
Issue
1
Pages
85-108
publication.type
International
Paper Link
Open Link
Supplementary Materials
Not Available
Abstract
Lip-reading technologies are rapidly progressing following the breakthrough of deep learning. It plays a vital role in its many applications, such as: human-machine communication practices or security applications. In this paper, we propose to develop an effective lip-reading recognition model for Arabic visual speech recognition by implementing deep learning algorithms. The Arabic visual datasets that have been collected contains 2400 records of Arabic digits and 960 records of Arabic phrases from 24 native speakers. The primary purpose is to provide a high-performance model in terms of enhancing the preprocessing phase. Firstly, we extract keyframes from our dataset. Secondly, we produce a Concatenated Frame Images (CFIs) that represent the utterance sequence in one single image. Finally, the VGG-19 is employed for visual features extraction in our proposed model. We have examined different keyframes: 10, 15, and 20 for comparing two types of approaches in the proposed model: (1) the VGG-19 base model and (2) VGG-19 base model with batch normalization. The results show that the second approach achieves greater accuracy: 94% for digit recognition, 97% for phrase recognition, and 93% for digits and phrases recognition in the test dataset. Therefore, our proposed model is superior to models based on CFIs input.
Staff Members - Benha University