TikTok’s live video feature 'encouraged sexual content', 'groomed' minors, Utah AG claims

TikTok has long known that its popular video livestreams encourage sexual content, including streams exploiting and “grooming” minors, according to a lawsuit by the state of Utah that was unredacted on Friday.

TikTok also discovered through an internal investigation that the feature, called TikTok Live, has facilitated money laundering and allowed users to sell drugs and fund terrorism, the lawsuit alleges.

A bipartisan group of attorneys general from more than a dozen states sued TikTok last fall, accusing the app of financially and psychologically harming minors. Utah also sued TikTok on similar grounds in June. Several of those lawsuits — including those from Vermont, New Hampshire, Kentucky and the District of Columbia — focused on TikTok Live. The suits cited investigative reporting from Forbes that found adult men have regularly used the livestreams to coax teen girls to perform racy, sometimes sexual acts in exchange for digital “gifts” that can be redeemed for money.

Utah and other states built their complaints on a trove of internal TikTok documents obtained via subpoena, but many of the specific allegations were redacted in public filings. Despite TikTok’s efforts to keep the records confidential, a Utah district court judge ruled on Dec. 19 to unseal much of the material in the lawsuit.

Utah’s newly unredacted complaint shows the company conducted a sweeping internal investigation into TikTok Live following the Forbes reporting. The investigation, codenamed Project Meramec, has not been previously reported. It confirmed the app facilitated potentially illegal financial transactions and problematic livestreams during which underage users would engage in suggestive or sexualized activities as viewers showered them with virtual gifts that could be converted to cash.

Project Jupiter

The complaint also details Project Jupiter, a separate internal investigation into the prevalence of money laundering on the platform. That investigation, which has also not been previously reported, found that TikTok Live and its gifting feature were being used to illegally launder money used to sell drugs and commit other crimes. That investigation was launched in 2021. TikTok employees have also discussed ways Live has been used to fund terrorist organizations overseas, including the Islamic State, the lawsuit claims.

“This lawsuit ignores the number of proactive measures that TikTok has voluntarily implemented to support community safety and well-being,” a TikTok spokesperson said in a statement. “Instead, the complaint cherry-picks misleading quotes and outdated documents and presents them out of context, which distorts our commitment to the safety of our community.”

The unredacted complaint comes just over two weeks before TikTok could be banned in the US under a national security law signed in April by President Joe Biden. The measure aims to address longstanding fears that the TikTok app, owned by China-based parent ByteDance Ltd., could be used to gather sensitive intelligence on Americans, or manipulate the content people see related to key political and social issues. TikTok’s popularity with minors, and the well-documented dangers it can pose to them, has only exacerbated those concerns.

Internal Presentations

Utah’s lawsuit highlights chat logs from TikTok employees, slides from internal presentations and other communications related to the Forbes report and Project Meramec, which found that hundreds of thousands of minors ages 13 to 15 were hosting livestreams and receiving troubling comments from adults, the lawsuit alleges.

“You’re paying my bills,” one TikToker, a 14-year-old in a bralette, told thousands of strangers on one livestream described in the Forbes report. The viewers, including grown men, offered to send the girl TikTok gifts and money for flashing the camera and showing them her feet. In another livestream with thousands of viewers, a young teen slowly cut off her t-shirt with scissors as commenters promised thousands of TikTok “coins,” which could translate to hundreds of dollars, if she’d strip and remove her bra. TikTok takes a financial cut when virtual gifts are shared on the app.

In the cases reviewed by Forbes, and also by TikTok through Project Meramec, users were too young to be allowed on the livestreams. At the time, TikTok’s policies forbade people under 16 from streaming live, and those under 18 from sending or receiving virtual gifts. Still, TikTok’s internal investigation found that 112,000 underage users hosted livestreams in a single month in 2022, the lawsuit alleges.

TikTok has since tightened its requirements for Live. Today, users must be at least 18 years old to use the feature.

TikTok also found that its algorithm boosted these types of sexualized videos so that they were more widely distributed. “Transactional sexual content hits most of business’ metrics of success & is pushed to TopLives because Algo prioritizes engagement,” said one internal slide from May 2022 that was mentioned in the lawsuit.

‘Significant Revenue’

Project Meramec found that young teen hosts were “at risk of grooming” on Live, that Live was bringing in “significant revenue from transactional gifting... primarily in exchange for [sexual] content,” that comments and direct messages were “used to make requests, harass hosts,” and that hosts regularly guided viewers to pay them on other platforms, the slide said.

Now, hosts can set spending limits for Live and tap their own moderators to help safeguard the broadcasts, including by filtering certain keywords or comments, or blocking or muting viewers. The livestreams are also monitored by TikTok’s own safety teams and detection systems.

Margaret Busse, executive director of Utah’s Department of Commerce, who sits on the cabinet of Utah Governor Spencer Cox, said that while the risks social media can pose to minors are well known, the revelations about TikTok Live – and the ways the dangers are multiplied when money is changing hands – add a new layer of complexity to the problem.

“When they understood what was happening, they also understood the amount of money it was making, and they did not want to do anything about it,” she said in an interview with Bloomberg. “That’s just what is so incredibly damning in my mind.”

See More: TikTok’s Algorithm Keeps Pushing Suicide to Vulnerable Kids

TikTok’s awareness of these issues, and its struggle to balance business goals with child safety, appear to have predated the Forbes investigation, which was published in 2022.

‘Limited Options’

In June 2021, according to the lawsuit, a senior TikTok employee raised alarm that “we have very limited options to protect minors from live content,” urging that “until we establish reliable content filtering strategies, we need to consider BLOCKING all L1s [13–15 year olds] from viewing (or interacting in any capacity) with Live.”

Internal chats between TikTok leaders in February 2022 cited in the suit show these concerns persisted.

“I do not understand why they cannot limit access to Live,” one leader wrote.

“I don’t know if we can reasonably expect to increase limitations for Live at this point,” another replied. “It is such a huge part of the strategy for the platform.”

“We can’t shut down access to levels that strangle the product’s ability to grow but we do need to use the signals that we have to limit access when we know the likelihood of harm or abusive behaviour is higher,” said another response. “Let’s try to find that balance.”

The company has emphasized its commitment to enforcing its policies and said it shut down more than 45 million streams last year.