Publications

2024

1. A multi-purpose audio-visual corpus for multi-modal Persian speech recognition: The Arman-AV dataset

   Javad Peymanfard, Samin Heydarian, Ali Lashini, and 3 more authors

   Expert Systems with Applications, 2024

   Abs Bib HTML PDF

   Automatic lip reading has advanced significantly in recent years. However, these methods need large-scale datasets that are scarce for many low-resource languages. In this paper, we introduce a new multipurpose audio-visual dataset for Persian. The dataset contains approximately 220 h of videos from 1760 speakers. The dataset can be used for multiple tasks, such as lip reading, automatic speech recognition, audio-visual speech recognition, and speaker recognition. It is also the first large-scale lip reading dataset in this language. We provide a baseline method for each task and propose a technique to identify visemes (visual units of speech) in Persian. The visemes obtained by this technique improve the accuracy of the lip reading task by 7% relatively compared to the previously proposed visemes, which can be generalized to other languages as well.

   @article{peymanfard2024multi,
     title = {A multi-purpose audio-visual corpus for multi-modal Persian speech recognition: The Arman-AV dataset},
     author = {Peymanfard, Javad and Heydarian, Samin and Lashini, Ali and Zeinali, Hossein and Mohammadi, Mohammad Reza and Mozayani, Nasser},
     journal = {Expert Systems with Applications},
     volume = {238},
     pages = {121648},
     doi = {10.1016/j.eswa.2023.121648},
     url = {https://www.sciencedirect.com/science/article/pii/S0957417423021504},
     year = {2024},
     publisher = {Elsevier}
   }

2022

1. Word-level Persian Lipreading Dataset

   Javad Peymanfard, Ali Lashini, Samin Heydarian, and 2 more authors

   2022

   Abs Bib HTML PDF

   Lip-reading has made impressive progress in recent years, driven by advances in deep learning. Nonetheless, the prerequisite such advances is a suitable dataset. This paper provides a new in-the-wild dataset for Persian word-level lip-reading containing 244,000 videos from approximately 1,800 speakers. We evaluated the state-of-the-art method in this field and used a novel approach for word-level lip-reading. In this method, we used the AV-HuBERT model for feature extraction and obtained significantly better performance on our dataset.

   @article{peymanfard2022word,
     title = {Word-level Persian Lipreading Dataset},
     author = {Peymanfard, Javad and Lashini, Ali and Heydarian, Samin and Zeinali, Hossein and Mozayani, Nasser},
     booktitle = {2022 12th International Conference on Computer and Knowledge Engineering (ICCKE)},
     pages = {225--230},
     doi = {10.1109/ICCKE57176.2022.9960105},
     url = {https://ieeexplore.ieee.org/abstract/document/9960105/},
     year = {2022},
     organization = {IEEE}
   }