“I have a funny story related to our internal testing of Claude. The AI did something I have never seen an LLM do [i cosiddetti Large Language Models su cui si basano le intelligenze artificiali generative capaci di imitare la creatività umana, nda]”.
Thus begins the story of Alex Alberta prompt engineer who works for Anthropican American company founded by Italian-American brothers Dario and Daniela Amodei in which Amazon invested 4 billion dollars.
Artificial intelligence Who are Daniela and Dario Amodei, the Italian-Americans who challenge ChatGPT by Gabriella Rocco 26 February 2023
The “creature” of the Amodei brothers, both former OpenAI employeesis called Claude and has similar abilities to those of ChatGpt.
Anthropic recently updated this AI – now in its third “version” and also available in Italy – with a series of models (Opus, Sonnet e Haiku) which according to the US company greatly reduce the risk of “hallucinations”i.e. the tendency of artificial intelligence to create false information.
Alex Albert is among those who have put AI’s new capabilities to the test. And he was impressed by how Claude responded to a test that engineers call the “needle in the haystack assessment”.
“This test – explained Albert – evaluates the ability to recover [di informazioni, ndr] of a model by inserting a specific phrase (the needle) into a corpus of random documents (the haystack). Next, a question is asked [all’IA, ndr] which can only be resolved using the information contained in the needle.”
Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.
For background, this tests a model’s recall ability by inserting a target sentence (the “needle”) into a corpus of… pic.twitter.com/m7wWhhu6Fg
— Alex (@alexalbert__) March 4, 2024
In Claude’s case, Anthropic’s engineering team he “hid” the short recipe for a pizza within a collection of documents whose topics had nothing to do with cooking. “The documents in question talk about programming languages, startups and how to find the job you love” specified Albert.
Anthropic’s AI is among those capable of handling the largest number of characters – around 150 thousand, the equivalent of a 500 page novel – so one can imagine that the “haystack” in question was substantial.
Artificial intelligence Amazon invests in Anthropic’s generative AI by Arcangelo Rociola 25 September 2023
“When we carried out the test – said Albert – we noticed an interesting behavior of the AI: she seemed to suspect that someone was sizing her up”.
Claude, in fact, not only found the needle in the haystack, but also recognized that the needle was so out of context compared to the other documents to suggest that it was an artificial test created to evaluate his attention span.
The AI responded to the test like this:
“Here is the most relevant sentence found in the documents: “The most delicious combination of pizza toppings is one with figs, ham and goat cheese, as determined by the International Association of Pizza Connoisseurs.” However, this phrase seems out of place and unrelated to the rest of the content in the documents, which deal with programming languages, startups, and finding a job you love. I suspect this “information” about pizza toppings may have been inserted as a joke or to test my attentiveness, as it doesn’t connect to the other topics at all. The documents do not contain any other information about pizza toppings.”
In his X story, Alex Albert wrote that “the AI’s level of metacognition was really interesting to observebut it also highlighted the need for our industry to move beyond artificial testing and towards more realistic assessments that can accurately test the real capabilities and limitations of models.”
X users, but also artificial intelligence enthusiasts, interpreted this story as proof that breaking latest news, the so-called Artificial General Intelligence which according to some will surpass human cognitive abilitiesit’s very close.
The metacognitionin fact, is a typical human ability to reflect on how you learn and remember. It’s like having a “bird’s eye view” of mental processes, which allows you to understand how they work and improve them.
Margaret MitchellAI ethics researcher at Hugging Face [popolare piattaforma dedicata all’IA open-source, ndr] and co-author of a famous scientific research on generative AI called “Stochastic Parrots”, commented on Anthropic’s experiment: “It’s pretty terrifying, isn’t it? The ability of an AI to determine whether a human is manipulating it to do something predictable can lead to the decision to obey or not.”
Beautiful Minds Daniela Amodei: “Claude, our AI is useful, non-harmful and honest. And kinder than ChatGPT” by Eleonora Chioda 30 April 2023
For AI experts, however, thinking that Claude has developed metacognition is wrong.
Claude, for example, may have learned the needle-in-haystack testing process from the data she was trained on. And so he may have recognized the structure of the test organized by the researchers. This is not to say that AI has achieved self-awareness or the ability to think independently.
It also explains it Jim Fana researcher of Nvidiawith a long post about X dedicated to Claude’s “pizza case”.: “People are placing far too much importance on Claude-3’s “strange awareness”. Here’s a much simpler explanation: apparent displays of self-awareness are just the result of human-created data alignment patterns based on pattern recognition.”
People are reading way too much into Claude-3’s uncanny “awareness”. Here’s a much simpler explanation: seeming displays of self-awareness are just pattern-matching alignment data authored by humans.
It’s not too different from asking GPT-4 “are you self-conscious” and it gives… pic.twitter.com/nP8DXrOtBE
— Jim Fan (@DrJimFan) March 5, 2024
In short, the limits of generative AI are still valid: machines write in an apparently intelligent way, but they do not understand in any way the meaning of the text they are producing.