New Tool Helps Creatives Fight Off AI Image Generators by ‘Poisoning’ Them
A new tool created by a team at the University of Chicago can add undetectable pixels to images to help creatives protect their work from AI image generators by effectively poisoning (corrupting) the AI’s training data.
As seen on Engadget, the tool, affectionately named Nightshade, was developed in response to massive companies such as OpenAI and Meta facing lawsuits for copyright infringement, with litigation alleging that their respective AI tools have stolen personal artworks without consent or compensation.
According to a report from the MIT Technology Review, Professor Ben Zhao and his team from the University of Chicago created Nightshade to help creatives fight back against Artificial Intelligence (AI) companies that use artists’ work to train their AI models without permission of the creators, or compensating them. The tool is currently under peer review but the team has already tested it against a recent Stable Diffusion model and an AI tool that the researchers built from scratch with very promising results.
The team hopes that the tool (which also leverages another tool called Glaze created by the same team) could be used to poison images/content shared online to protect it and “damage” future iterations of image-generating AI models like DALL-E, Midjourney, and Stable Diffusion.
The “poison” basically alters how machine-learning tools interpret the data it scrapes from online sources so that it sees and reproduces something entirely different. The altered pixels are invisible to the human eye, but are completely manipulated when viewed by an AI Model. An example shown by Zhao’s team goes so far as to include an original image of a car, with the “interpreted image” by the AI model resulting in a cow.
Using this tool, artists who want to share their work online but still protect their images can upload their work into Glaze and enable Nightshade to apply its AI poison. According to Zhao, Nightshade and Glaze will be free to use, with Nightshade being open-source to allow for additional improvements, with the hope that if enough people start using it to poison their content from AI Models, it will encourage bigger companies to properly compensate and credit original artists.
“The more people use it and make their own versions of it, the more powerful the tool becomes,” Zhao says. “The data sets for large AI models can consist of billions of images, so the more poisoned images can be scraped into the model, the more damage the technique will cause.”
“Poisoned data samples can manipulate models into learning, for example, that images of hats are cakes, and images of handbags are toasters. The poisoned data is very difficult to remove, as it requires tech companies to painstakingly find and delete each corrupted sample.” Nightshade would not only infect the AI-trained word “dog” but also all similar concepts such as puppy, husky, beagle, and wolf.
Zhao admits that there is a possibility that people will also abuse the data poisoning tool for malicious uses, but those people would need thousands and thousands of poisoned samples to inflict any real damage on a larger scale.
A preview article called Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models was published on arXiv which outlines the scope and functionality of Nightshade for those looking to dive a little deeper into how the tool works.
Image credits: Body images by Professor Ben Zhao / University of Chicago. The featured image was created by PetaPixel using an AI image generation model.
