Spoken Utterance Classification Task of Arabic Numerals and Selected Isolated Words

Nowadays, sensory organs are becoming essential means for controlling modern machines which require human intervention. Among these means, we can cite the sense of voice which can be used to control and monitor modern interfaces. In this regard, Automatic Speech Recognition (ASR) is mainly explored to accomplish many tasks, such as translating natural voice into computer text and performing actions based on human commands. In this paper, a system for recognizing spoken Arabic numerals and words based on two classification methods is proposed. The first classification approach is a combination of Convolutional Neural Network (CNN) with Long Short-Term Memory (LSTM) and Fully Connected (FC) network (CNN-LSTM-FC), while the second is based on the conventional Dense Network (DenseNet). These classification approaches are integrated into the proposed Arabic speech recognition system to perform the classification task by exploring uniform length sequences of speech utterances extracted from the Mel-frequency Cepstral Coefficients (MFCCs). Regarding the CNN-LSTM-FC approach, it is offered with the objective of learning high-level features that contain long-term contextual dependencies and local information. These features include less information than raw data, which helps to reduce the training time. Also, the CNN-LSTM-FC method allows capturing global contextual information and local correlation results from MFCC coefficients. With respect to the DenseNet model, it is explored to benefit from the direct connections \(\frac{{L\left( {L + 1} \right)}}{2}\) between its layers in addition to its ability to alleviate the problem of the vanishing of gradient and the reduction in the number of its explored parameters. The training time is therefore reduced. Our models were evaluated on two databases: The first is a database of English voice commands, while the second is that of spoken Arabic numerals and words. Experimental tests showed that the CNN-LSTM-FC model with MFCC coefficients performed best on the database of spoken Arabic numerals and words in terms of evaluated performances (accuracy = 88.04%, precision = 88.56%, recall = 87.78%, F1 = 88.17, and error = 1.10%) compared to those obtained with the DenseNet model. Additionally, the best results on the database of English voice command for precision (87.15%), F1 (85.66), and error (0.58%) were obtained by the CNN-LSTM-FC model, while those for accuracy (85.40%) and recall (85.40%) were achieved using the DenseNet model. Even the two proposed models led to acceptable results on both databases; however, they require less computation to achieve higher performance.