Visual speech recognition (VSR) has received increasing attention in recent decades due to its potential uses in many applications. As for any recognition system, useful materials for training and testing are required. For VSR system development, the training and testing materials are videos representing the visual speech of the words. This paper presents the Arabic Visual Speech Dataset (AVSD) for visual speech recognition. The dataset contains 1100 videos for 10 daily communication words collected from 22 speakers and recorded using smartphones’ cameras in high-resolution and high-framerate. The process of building the dataset, including design, acquisition, post-processing phases are described in the paper. Finally, the results of evaluating AVSD using a VSR system are presented and discussed.