IISc's Project Vaani Creates Atypical Speech Dataset for Three Indian Languages

Researchers at the Indian Institute of Science (IISc), Bengaluru, have developed a pilot dataset of atypical speech in three Indian languages under Project Vaani. The initiative, called the Vaani Atypical Speech Corpus, aims to make automatic speech recognition (ASR) systems more inclusive for people with speech impairments.

Atypical speech refers to speech patterns that deviate from standard speech, often due to neurological, cognitive, or motor conditions. While ASR technology has advanced for standard speech, it struggles with atypical speech, particularly in Indian languages where such data is scarce.

The pilot corpus currently contains approximately 10 hours of atypical speech from around 40 speakers across three languages. The team plans to expand the dataset in the coming months. The project is a collaboration with Google's Project Euphonia, which focuses on improving speech recognition for people with atypical speech.

“For Indian languages, there is no such data,” said Prasanta Kumar Ghosh, a professor in the Department of Electrical Engineering at IISc who leads Project Vaani. He emphasized the need to create technology that serves all individuals, including those who cannot speak naturally.

The larger Project Vaani aims to capture India's linguistic diversity by building a corpus of 150,000 hours of natural speech from around 1 million people across nearly 800 districts. So far, the team has completed two phases covering 165 districts, 156,534 speakers, and 105 languages. They have recorded over 31,000 hours of speech data, which is open-sourced.

Data collection uses an image-prompted method where speakers describe images shown to them. The team uses generic and locality-specific images to capture multimodal information. Each speaker spends about 20 seconds per image, with a limit of 60 images per speaker to ensure variety. The team also strives for balance in gender, age, socio-economic status, and education.

Around 10% of the collected data has been transcribed, amounting to more than 2,000 hours of transcription. The atypical speech pilot is a step toward making speech technology accessible to all Indians, regardless of speech patterns.