What if it existed a system to cheat artificial intelligence? A system that can cheat the training of the so-called LLMs, large language models, and any other algorithm trained on the basis of incredible amounts of data collected online?
Indeed there is, and for now it concerns works of art and those of creativity. It is still a sort of experiment in civil disobedience, or rather an attempt to assert one’s rights by directly influencing the fuel on which these generative models are developed. It’s called Nightshade e Ben Zhao, a professor, invented it from the University of Chicago.
The story Privacy and facial recognition: the adversative clothes of an Italian startup will hide us from Emanuele Capone’s AI 26 August 2022
Nightshadewhich will soon be integrated into another sister tool, allows you to edit minimally and imperceptible to the human eye the pixels of the images before uploading them online to sites, social networks and other platforms. By incorporating a message that AI systems will read differently: if and when that file is included in an AI training dataset of some hi-tech giant or some specialized startup, it might be able to sabotage the model from within. Producing outcomes opposite to what developers and users might expect.
There is clearly an underlying political intent: to fight against companies specializing in AI (or that are developing powerful and large-scale models) that collect everything from the Web without being too subtle (it’s called scraping and we had explained it here). And in particular in the so-called Text-to-Image models, those such as Midjourney or Dall-E 3 by OpenAI who generate images or illustrations based on simple textual input and who are hungry for images, works of art, artistic creations and other such content. They serve precisely as a training platform to make the systems increasingly precise, capable of reproducing different styles and approaches.
In a sense, Nightshade poisons the wells, compromising the source data and holding the potential (if the large-scale artistic community would employ this tool) to create a lot of problems for Stable Diffusion and company. To clarify, dogs become cats, cars look like cows and so on, thanks to the information contained in the modified pixels they convey different information compared to the actual content of the image. At least in the eyes of machines.
An initial field test of the system was presented for review at the Usenix cybersecurity conference: MIT Technology Review previewed it discovering how, both tested on an internally developed AI and on Stable Diffusion, even a relatively low number of such compromised images can create significant headaches. On the other hand, the groups working on the topic, from OpenAI to Meta, via Google and Stability AI, are grappling all over the world with a series of lawsuits brought by artists and creatives (among the first, last January, that of Sarah Andersen, Kelly McKernan and Karla Ortiz), which accuse these companies of using copyrighted material and personal information without asking permission or offering compensation. For Zhao, Nightshade could be a deterrent.
The expert has among other things also developed Glaze, another tool that allows you to mask the style of a work. Therefore not to modify the metadata it conveys but to pass inconsistent information on the style of the work to the algorithm. So as to ensure that a user, with a poorly written input line, can easily reproduce works in the style of some artist, even unconsciously. Also in this case, modifying the pixels of the images in ways not perceptible to the human eye, this content is able to manipulate machine learning models to interpret the image as something other than what it actually shows.
In-depth analysis Generative AI has a problem with female beauty, but it’s not (only) its fault by Francesco Marino 07 October 2023
The development team wants integrare Nightshade in Glaze and make this package open source, so that other versions can be developed. The belief is that the more artists use these systems, the more significant their impact will become in the eyes of the giants who are developing generative AI models based on billions of images. Also because every corrupted element, that is, every image passed through the hands of Nightshade, should be identified and removed individually.
Obviously there is also no shortage of risks associated with these techniques can be abused, obviously: “We still don’t know if (companies, ed.) have robust defenses against these attacks. We have not yet seen attacks of this kind against modern machine learning models, but it could only be a matter of time,” he commented Vitaly Shmatikov, professor at Cornell University who studies the security of AI models and was not involved in the project – This is the time to work on defenses.”