Make some noise for love using AI

For many, the online space is unsafe as queer-phobic and casteist slurs are often made in vernacular Indian languages, which most reporting systems are not trained in. Shhor AI, a unique tool that uses AI to detect hate speech in Hinglish and the Indian context, hopes to set the right tone, writes Aisiri Amin.

In 2021, political artist Arpita posted an artwork on her Instagram page @blabberville showing two people talking about how horrible the Holocaust was but then watching news about Palestinians getting killed in Gaza while one asks the other to switch the channel. The Instagram post was immediately flooded with comments calling her anti-Semitic and her message folder was filled with threats and hate.

“Threats and constant use of abusive language can take a toll on our mental health,” says Arpita. So, when she came across Shhor AI, an artificial intelligence (AI) bot that detects and removes online hate speech, it felt like hope with teeth.

Shhor AI, created by AI engineer and 27-year-old artist Aindriya Barua (who uses the pronouns they/them), stemmed from their experience with constant hate speech received for their artwork on social media. Its special focus on vernacular languages and the Indian context makes Shhor AI a one-of-a-kind tool to address hate speech.

Rooted in personal experiences

As a neurodivergent, Indigenous, and queer person, finding a community where they felt safe had always seemed like a tough task for Barua. So, art had become a way of self-expression and asserting their place in the world.

When sharing their artwork on social media in 2019 came with a sense of belonging, they also received hate through trollers and people with extremist views who immediately dived into threats and hate speech.

Barua says they didn’t choose to be a political artist; there was just no other way. “When you talk about your marginalised identities, everything becomes political including the art you create,” they explain.

The online hate often percolated into the real world. For instance, when in 2021 Barua posted an artwork about the violation of human rights under the Armed Forces Special Powers Act (AFSPA), the threats no longer remained online.

“I remember an autorickshaw parked outside my house was constantly playing a song with violent lyrics through speakers all day,” they say.

It was then they decided to use their skills as an AI engineer and research experience in natural language processing (NLP) to find a solution. As the hype around AI was beginning in 2020, questions emerged in Barua’s mind — if AI has so much power, why don’t we build tools to help marginalised people? Why was I building software for the privileged?

Building Shhor AI

In 2021, Barua decided to start building the AI tool and named it Shhor AI because ‘shor’ means uproar in Urdu. “It is a project to amplify our voices. The ‘Shh’ in the Shhor AI stands for shushing the hate and making noise for love,” they add.
The project’s core was the understanding that a content-moderation technology built with a Global North, white, or saviour glance of the privileged, wouldn’t understand our specific nuanced issues, Barua says. “In India, people often use an English keyboard to write vernacular languages we speak in. For instance, someone might write a Hindi slur word in English which makes it difficult to detect. So, there are no fixed spellings or grammar. This is called code-mixing,” they explain.

Instagram does allow you to block certain words or remove comments but it requires a lot of manual effort. For instance, Arpita has used the feature to stop some words from popping up in her comments section. “But how many words can one block manually and in how many languages? People also use new derogatory comments based on current events; it’s a difficult task,” Arpita adds.

To address these issues and more, Barua decided to weaponise the abuse that they and other marginalised artists were receiving.

They created a scraper to automatically collect hate comments from people who shared links to their targeted posts.

Barua then read through thousands of comments to mark them as hate speech or not and soon some people joined in to support this daunting process.

With this, they have created “an extensive Hinglish real-world dataset with more than 50,000 tagged comments,” Barua says. This data was then used to train the AI model.

Shhor AI is trained to understand the context of the sentences and detect hate essence even when written in Hindi using an English keyboard or when people use Hinglish.

It is also unique because it can identify hate speech even when symbols are used to write words such as replacing ‘a’ with ‘@’.

Shhor AI also understands intersectionality. For instance, when Barua collected data on queerphobia, it showed that a Muslim queer person was getting more hate. So, they knew it was not possible to look at hate speech without understanding the intersections of marginalised identities.

How Shhor AI works

In 2022 Barua tested an AI bot based on Shhor AI’s application programming interface (API) for Reddit. The bot, when active on a subreddit, successfully detected hate speech and banned people if they continued to use hate speech after three warnings.

Barua’s tool won the United Nations Populations Fund International Hackathon on solving technology-facilitated gender violence (TF-GBV) for Shhor AI in 2022. Earlier this year, Shhor AI won the Just AI Awards 2024 in the category ‘AI for Good’ at the Global Summit 2024 held in Hyderabad. When asked if awards help, Barua says it opens the doors. “Marginalised people don’t have the privilege of connections or platforms to upscale. We don’t have a big grant or generational wealth, so the hope is that visibility through awards or recognitions makes way for funding,” they explain.

Hope for an inclusive space

People have reported to Shhor about biases experienced in Meta’s content moderation, they add. “When they talk about certain political issues, sometimes their post is removed, or their reach is reduced. Even if an emoji or a different spelling is used, such posts are detected,” Barua says.

So, a centralised system is not the solution. Barua emphasises that every country, every language needs their content moderation tools customised with inputs from the people of the grassroots with lived experience to define what is hate speech for them.

Barua calls themselves angry but hopeful.

“After all, revolution is the love child of rage and hope,” they say. “The only reason we fight is because we have hope for a better future.”