Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language
IEEE Access • 2021
Publication Information
Authors
Farman Shah, Muhammad Saqlain Shah, Waseem Akram, Awais Manzoor, Rasha Orban Mahmoud, And Diaa Salama Abdelminaam
Keywords
Sign language, image recognition, machine learning, features extraction
Journal
IEEE Access
Publisher
Not Available
Volume
82
Issue
Not Available
Pages
105565
publication.type
International
Paper Link
Open Link
Supplementary Materials
Not Available
Abstract
All over the world, deaf people use sign language as the only reliable source of communication with each other as well as with
normal people. These communicating signs are made up of the shape of the hand and movement. In Pakistan, deaf people use
Pakistan sign language (PSL) as a means of communication with people. In scientific literature, many studies have been done on
PSL recognition and classification. Most of this work focused on colored-based hands while some others are sensors and Kinectbased approaches. These techniques are costly and also avoid user-friendliness. In this paper, a technique is proposed for the
recognition of thirty-six static alphabets of PSL using bare hands. The dataset is obtained from the sign language videos. At a later
step, four vision-based features are extracted i.e., local binary patterns, a histogram of oriented gradients, edge-oriented
histogram, and speeded up robust features. The extracted features are individually classified using Multiple kernel learning (MKL)
in support vector machine (SVM). We employed a one-to-all approach for the implementation of basic binary SVM into the multi-
class SVM. A voting scheme is adopted for the final recognition of PSL. The performance of the proposed technique is measured
in terms of accuracy, precision, recall, and F-score. The simulation results are promising as compared with existing approaches.
normal people. These communicating signs are made up of the shape of the hand and movement. In Pakistan, deaf people use
Pakistan sign language (PSL) as a means of communication with people. In scientific literature, many studies have been done on
PSL recognition and classification. Most of this work focused on colored-based hands while some others are sensors and Kinectbased approaches. These techniques are costly and also avoid user-friendliness. In this paper, a technique is proposed for the
recognition of thirty-six static alphabets of PSL using bare hands. The dataset is obtained from the sign language videos. At a later
step, four vision-based features are extracted i.e., local binary patterns, a histogram of oriented gradients, edge-oriented
histogram, and speeded up robust features. The extracted features are individually classified using Multiple kernel learning (MKL)
in support vector machine (SVM). We employed a one-to-all approach for the implementation of basic binary SVM into the multi-
class SVM. A voting scheme is adopted for the final recognition of PSL. The performance of the proposed technique is measured
in terms of accuracy, precision, recall, and F-score. The simulation results are promising as compared with existing approaches.
Staff Members - Benha University