Due to the significance of ancient Arabic manuscripts and their role in enriching valuable historical information, this study aims to collect Arabic manuscripts in a dataset and classify its images to predict their authors. We accomplished this study through two main phases. First is the data collection phase. Arabic manuscripts gathered, including 52 Arabic Authors. Second is the models’ development phase to extract the visual features from the images and train the networks on them. We built four deep learning models named: MobileNetV1, DenseNet201, ResNet50, and VGG19. We configured the models by tuning their learning hyperparameters toward optimizing their recognition process. Afterward, we performed a comparative analysis between all the models to measure their performance. Eventually, we reached that minimizing the learning rate, combining “Sigmoid” with “Softmax”, and increasing the number of neurons on the final classification dense layer improved the networks’ recognition performance significantly since all utilized deep learning models reached above 95% validation accuracy.