This tool could protect artists from AI that steals their style

Robots would come for humans’ jobs. That was guaranteed. The assumption generally was that they would take over manual labour, lifting heavy pallets in a warehouse and sorting recycling.

Now significant advances in generative artificial intelligence mean robots are coming for artists, too.

AI-generated images, created with simple text prompts, are winning art contests, adorning book covers, and promoting The Nutcracker, leaving human artists worried about their futures.

Read | The rise of AI art

The threat can feel highly personal. An image generator called Stable Diffusion was trained to recognise patterns, styles and relationships by analysing billions of images collected from the public internet, alongside text describing their contents.

Among the images it trained on were works by Greg Rutkowski, a Polish artist who specialises in fantastical scenes featuring dragons and magical beings.

Seeing Rutkowski’s work alongside his name allowed the tool to learn his style effectively enough that when Stable Diffusion was released to the public last year, his name became shorthand for users who wanted to generate dreamy, fanciful images. One artist noticed that the whimsical AI selfies that came out of the viral app Lensa had ghostly signatures on them, mimicking what the AI had learned from the data it trained on: artists who make portraits sign their work. “These databases were built without any consent, any permission from artists,” Rutkowski said.

Since the generators came out, Rutkowski said he has received far fewer requests from first-time authors who need covers for their fantasy novels. Meanwhile, Stability AI, the company behind Stable Diffusion, recently raised $101 million from investors and is now valued at over $1 billion.

“Artists are afraid of posting new art,” computer science professor Ben Zhao said. Putting art online is how many artists advertise their services but now they have a “fear of feeding this monster that becomes more and more like them,” Professor Zhao said. “It shuts down their business model.”

That led Professor Zhao and a team of computer science researchers at the University of Chicago to design a tool called Glaze that aims to thwart AI models from learning a particular artist’s style. To design the tool, which they plan to make available for download, the researchers surveyed more than 1,100 artists and worked closely with Karla Ortiz, an illustrator and artist based in San Francisco.

Say, for example, that Ortiz wants to post new work online, but doesn’t want it fed to AI to steal it. She can upload a digital version of her work to Glaze and choose an art type different from her own, say abstract.

The tool then makes changes to Ortiz’s art at the pixel level that Stable Diffusion would associate with, for example, the splattered paint blobs of Jackson Pollock.

To the human eye, the Glazed image still looks like her work, but the computer-learning model would pick up on something very different. It’s similar to a tool the University of Chicago team previously created to protect photos from facial recognition systems.

When Ortiz posted her Glazed work online, an image generator trained on those images wouldn’t be able to mimic her work. A prompt with her name would instead lead to images in some hybridised style of her works and Pollock’s.

Read | Clickbait or creativity? The art world wrestles with AI

“We’re taking our consent back,” Ortiz said. AI-generating tools, many of which charge users a fee to generate images, “have data that doesn’t belong to them,” she said. “That data is my artwork, that’s my life. It feels like my identity.”

The team at the University of Chicago admitted that their tool does not guarantee protection and could lead to countermeasures by anyone committed to emulating a particular artist. “We’re pragmatists,” Professor Zhao said. “We recognise the likely long delay before law and regulations and policies catch up. This is to fill that void.”

Many legal experts compare the debate over the unfettered use of artists’ work for generative AI to pirating concerns in the early days of the internet with services like Napster that allowed people to consume music without paying for it. The generative AI companies are already facing a similar barrage of court challenges.

Last month, Ortiz and two other artists filed a class-action lawsuit in California against companies with art-generating services, including Stability AI, asserting violations of copyright and right of publicity.

“The allegations in this suit represent a misunderstanding of how generative AI technology works and the law surrounding copyright,” the company said in a statement. Stability AI was also sued by Getty Images for copying millions of photos without a license. “We are reviewing the documents and will respond accordingly,” a company spokeswoman said.

Jeanne Fromer, a professor of intellectual property law at New York University, said the companies may have a strong fair use argument. “How do human artists learn to create art?” Professor Fromer said.

“They’re often copying things and they’re consuming lots of existing artwork and learning patterns and pieces of the style and then creating new artwork. And so at a certain level of abstraction, you could say machines are learning to make art the same way.”

At the same time, Professor Fromer said, the aim of copyright law is to protect and encourage human creativity. “If we care about protecting a profession,” she said, “or we think just the making of the art is important to who we are as a society, we might want to be protective of artists.”

A non-profit called the Concept Art Association recently raised over $200,000 through GoFundMe to hire a lobbying firm to try to persuade Congress to protect artists’ intellectual property. “We are up against the tech giants with unlimited budgets, but we are confident that Congress will recognise that protecting IP is the right side of the argument,” said the association’s founders, Nicole Hendrix and Rachel Meinerding.

It is possible that artists might get paid a nominal amount when their art is used to train AI and inspire new images, similar to how musicians get paid by music-streaming companies.

Andy Baio, a writer and technologist who examined the training data used by Stable Diffusion, said these services can mimic an artist’s style because they see the artist’s name alongside their work over and over again. “You could go and remove names from a data set,” Baio said, to prevent the AI from explicitly learning an artist’s style.

One service already seems to have done something along these lines. When Stability AI released a new version of Stable Diffusion in November, it had a notable change: the prompt “Greg Rutkowsi” no longer worked to get images in his style, a development noted by the company’s chief executive Emad Mostaque.

Stable Diffusion fans were disappointed. “What did you do to greg,” one wrote on an official Discord forum frequented by Mr. Mostaque. He reassured users of the forum that they could customize the model. “Training on greg won’t be too difficult,” another person responded.Mr. Rutkowski said he planned to start Glazing his work.