New AI system to help counter online hate speech

Scientists have developed an artificial intelligence (AI) system that could help counter hate speech directed at disenfranchised minorities such as the Rohingya community.

The system developed by researchers from Carnegie Mellon University in the US can rapidly analyse thousands of comments on social media, and identify the fraction that defends or sympathise with voiceless groups.

Human social media moderators, who could not possibly manually sift through so many comments, would then have the option to highlight this "help speech" in comment sections, the researchers said.

"Even if there's lots of hateful content, we can still find positive comments," said Ashiqur R KhudaBukhsh, a post-doctoral researcher at Carnegie Mellon University's Language Technologies Institute (LTI).

Finding and highlighting these positive comments might do as much to make the internet a safer, healthier place as would detecting and eliminating hostile content or banning the trolls responsible, the researchers said.

The Rohingyas, who began fleeing Myanmar in 2017 to avoid ethnic cleansing, are largely defenceless against online hate speech, they said.

Many of them have limited proficiency in global languages such as English, and they have little access to the internet.

Most are too busy trying to stay alive to spend much time posting their own content, KhudaBukhsh said.

To find relevant help speech, the researchers used their technique to search for more than a quarter of a million comments from YouTube in what they believe is the first AI-focused analysis of the Rohingya refugee crisis.

The ability to analyse such large quantities of text for content and opinion is possible because of recent major improvements in language models, said Jaime Carbonell, LTI director and a co-author on the study.

These models learn from examples so they can predict what words are likely to occur in a given sequence, and help machines understand what speakers and writers are trying to say.

However, the researchers developed a further innovation that made it possible to apply these models to short social media texts in South Asia.

Short bits of text, often with spelling and grammar mistakes, are difficult for machines to interpret.

It's even harder in South Asian countries, where people may speak several languages and tend to "code switch," combining bits of different languages and even different writing systems in the same statement.

Existing machine learning methods create representations of words, or word embeddings so that all words with a similar meaning are represented in the same way, the researchers explained.

This technique makes it possible to compute the proximity of a word to others in a comment or post.

To extend this technique to the challenging texts of South Asia, the team obtained new embeddings that revealed language groupings or clusters.

This language identification technique worked as well or better than commercially available solutions, the researchers said.

This innovation has become an enabling technology for computational analyses of social media in that region, Carbonell noted.

Samplings of the YouTube comments showed about 10 per cent of the comments were positive.

When the researchers used their method to search for help speech in the larger dataset, the results were 88 per cent positive, indicating that the method could substantially reduce the manual effort necessary to find them, KhudaBukhsh said.

The researchers presented their findings at the Association for the Advancement of Artificial Intelligence annual conference in New York City, US.