IIT-M faculty develop AI models to process Indian languages

Faculty at the Indian Institute of Technology Madras (IIT-M) have developed Artificial Intelligence (AI) models and datasets to process texts in 11 Indian languages, the premier institute said on Tuesday.

The initiative was taken up jointly with "AI4Bharat," a platform for building AI solutions for problems of relevance to the country, a release from IIT-M said here. The open-source tool, completely free of cost, can be downloaded from https://indicnlp.ai4bharat.org/.

"The multilingual AI models and datasets developed through this initiative will provide the essential building blocks to students, faculty, start-ups and industry to work on Indian language tools and push the frontiers of technology," it said.

Researchers from IIT Madras and AI4Bharat released AI models and datasets for Tamil, Hindi, Malayalam, Telugu, Kannada, Punjabi, Bengali, Odia, Assamese, Gujarati and Marathi. According to Mitesh M Khapra, Assistant Professor, Department of Computer Science and Engineering, IIT-M, as the country moves towards a digital economy, it is important that Indian languages find a space online.

"This requires a lot of innovation in creating input tools, datasets, and AI models for Indian languages," he said. "For example, imagine a learner who posts a question on an e-learning platform in Tamil or Hindi or any other numerous Indian regional languages. There is a need for tools that can automatically process such questions written in Indian languages and classify them into specific topics," he said.

Such tools were already available for English and other foreign languages but not for Indian ones, Khapra added. AI4Bharat is an initiative co-founded by Khapra and Pratyush Kumar, Assistant Professor, Department of Computer Science and Engineering, IIT Madras and works to solve India-specific problems in a community-driven, open-sourced manner, the release added.

Kumar said the initiative "is one of the few attempts in academia" to develop and publicly release large scale multilingual AI models containing millions of parameters trained on billions of tokens from 11 Indian languages, completely free and open-source.