<p>The recent X controversy has brought home to netizens the terrifying fallout of the unethical use of technology. The platform’s AI tool, Grok, is in the middle of a social media storm, prompting the Central Government to step in and direct X to take down offensive content. However, the government must also focus on bringing out a more holistic policy to regulate the use of generative AI, Bengaluru-based experts say.</p>.<p>Explicit images of women users on the microblogging site are being generated by other users by feeding Grok a simple prompt. While morphed and deep fake images have been doing the rounds for years, the ease with which this is possible now is alarming. (See box)</p>.Grok AI row: Govt seeks more details on action taken from X.<p><strong>Suo moto cases booked</strong></p>.<p>Seemant Kumar Singh, Bengaluru City Police Commissioner, says while there has not been a significant increase in such cases lately, it is quite concerning. They have been cracking down on the menace to keep such activity in check. “Even if not registered, we have been taking up suo moto cases,” he says, adding, “It is not difficult for us to trace the culprit.”</p>.<p>Agencies first try to take down offensive posts, and then find out who has done it and their motive.</p>.<p>The Bengaluru City Police also took to X to share awareness videos on the negative impact of generative AI.</p>.<p><strong>What law says</strong></p>.<p>According to Manavi Atri, human rights lawyer, it violates Sections 75 and 79 of the Bharatiya Nyaya Sanhita (BNS). Section 75 deals with sexual harassment, with culprits facing up to three years in prison or a fine, or both. Section 79 criminalises insulting a woman’s modesty through “word, gesture or act”. The punishment is imprisonment for up to three years, plus a fine.</p>.<p>Lawyer Ashok V G points out that it can also trigger liability under multiple statutes, including provisions on obscenity and unlawful dissemination under the BNS, the Information Technology Act, and the Indecent Representation of Women (Prohibition) Act.</p>.<p>He believes there is a fine line between artistic freedom and criminal conduct, and that regulating AI-driven abuse requires a holistic approach. He notes that measures such as mandating clear AI-generated labels, ensuring traceability of authorship for unlawful content, and strengthening law-enforcement capacity to investigate and remedy such offences are as essential as platform accountability.</p>.<p>“Denmark has adopted laws that extend copyright-style protection to individuals over their own features — an early step towards empowering victims to counter deepfakes. India must similarly consider proactive regulatory interventions to address a rapidly evolving phenomenon,” he states.</p>.<p>India’s AI policy remains largely aspirational, with more emphasis on innovation than on rights, safety, and due process, notes Apar Gupta, a digital rights activist. What is missing is a binding legal framework that sets clear obligations, independent oversight, and consequences for harm, he elaborates.</p>.<p>“Users should limit the personal data they share and be cautious about engaging with AI systems that lack transparency,” he tells Metrolife. However, he clarifies that individual vigilance cannot substitute for systemic protections, which is why regulatory accountability remains essential.</p>.<p><strong>What happened</strong></p>.<p>Towards mid-to-late December 2025, X users started misusing its AI chatbot, Grok, to generate non-consensual sexualised images of other female users, and in some cases even children. As these posts became viral, governments across the world came down strongly on the social media platform. In India, the Ministry of Electronics and Information Technology directed X on January 2 to remove all unlawful content and submit an action taken report within 72 hours. Though X acknowledged that the content was problematic and offered to permanently disable accounts of errant users generating obscene content, it has been reported that the government is dissatisfied with X’s ‘inadequate’ response and has sought further clarifications.</p>
<p>The recent X controversy has brought home to netizens the terrifying fallout of the unethical use of technology. The platform’s AI tool, Grok, is in the middle of a social media storm, prompting the Central Government to step in and direct X to take down offensive content. However, the government must also focus on bringing out a more holistic policy to regulate the use of generative AI, Bengaluru-based experts say.</p>.<p>Explicit images of women users on the microblogging site are being generated by other users by feeding Grok a simple prompt. While morphed and deep fake images have been doing the rounds for years, the ease with which this is possible now is alarming. (See box)</p>.Grok AI row: Govt seeks more details on action taken from X.<p><strong>Suo moto cases booked</strong></p>.<p>Seemant Kumar Singh, Bengaluru City Police Commissioner, says while there has not been a significant increase in such cases lately, it is quite concerning. They have been cracking down on the menace to keep such activity in check. “Even if not registered, we have been taking up suo moto cases,” he says, adding, “It is not difficult for us to trace the culprit.”</p>.<p>Agencies first try to take down offensive posts, and then find out who has done it and their motive.</p>.<p>The Bengaluru City Police also took to X to share awareness videos on the negative impact of generative AI.</p>.<p><strong>What law says</strong></p>.<p>According to Manavi Atri, human rights lawyer, it violates Sections 75 and 79 of the Bharatiya Nyaya Sanhita (BNS). Section 75 deals with sexual harassment, with culprits facing up to three years in prison or a fine, or both. Section 79 criminalises insulting a woman’s modesty through “word, gesture or act”. The punishment is imprisonment for up to three years, plus a fine.</p>.<p>Lawyer Ashok V G points out that it can also trigger liability under multiple statutes, including provisions on obscenity and unlawful dissemination under the BNS, the Information Technology Act, and the Indecent Representation of Women (Prohibition) Act.</p>.<p>He believes there is a fine line between artistic freedom and criminal conduct, and that regulating AI-driven abuse requires a holistic approach. He notes that measures such as mandating clear AI-generated labels, ensuring traceability of authorship for unlawful content, and strengthening law-enforcement capacity to investigate and remedy such offences are as essential as platform accountability.</p>.<p>“Denmark has adopted laws that extend copyright-style protection to individuals over their own features — an early step towards empowering victims to counter deepfakes. India must similarly consider proactive regulatory interventions to address a rapidly evolving phenomenon,” he states.</p>.<p>India’s AI policy remains largely aspirational, with more emphasis on innovation than on rights, safety, and due process, notes Apar Gupta, a digital rights activist. What is missing is a binding legal framework that sets clear obligations, independent oversight, and consequences for harm, he elaborates.</p>.<p>“Users should limit the personal data they share and be cautious about engaging with AI systems that lack transparency,” he tells Metrolife. However, he clarifies that individual vigilance cannot substitute for systemic protections, which is why regulatory accountability remains essential.</p>.<p><strong>What happened</strong></p>.<p>Towards mid-to-late December 2025, X users started misusing its AI chatbot, Grok, to generate non-consensual sexualised images of other female users, and in some cases even children. As these posts became viral, governments across the world came down strongly on the social media platform. In India, the Ministry of Electronics and Information Technology directed X on January 2 to remove all unlawful content and submit an action taken report within 72 hours. Though X acknowledged that the content was problematic and offered to permanently disable accounts of errant users generating obscene content, it has been reported that the government is dissatisfied with X’s ‘inadequate’ response and has sought further clarifications.</p>