Artificial intelligence is only as smart as its training data. Fed with data sets from billions of text documents and images, it learns the connections between concepts and visual representations in a training process that often lasts weeks. But what if an AI didn’t have all of this data available? What if she had to learn like a toddler who is gradually making sense of the world? That’s what researchers at New York University wanted to find out. Their study has now been published in the specialist magazine “Science”.
Advertisement
Between the ages of 1.5 and two years, children can understand up to 300 words on average. In order to find out exactly how the learning process works, the team led by cognitive scientist Wai Keen Vong equipped a single child with a small forehead camera for a short period of time over a period of almost a year and a half and recorded the first-person perspective. This gave the researchers a unique look at how a child learns what a ball, a cat or a bucket is through their eyes and the supportive vocalizations of those around them, such as their parents.
A total of 61 hours of video material was collected, which the researchers broke down into 600,000 video frames and 37,500 correlating vocalizations. They then used this data to train a “relatively generic neural network” called CVCL: Child’s View for Contrastive Learning. They wanted to find out whether a neural network, i.e. an AI, is able to learn what a ball or a cat looks like independently and exclusively using the toddler’s audiovisual information.
Learning with “minimal ingredients”
Although the recordings totaled only less than one percent of the child’s waking periods during the study period, the CVCL model was able to learn word-related associations. If the model was trained with the correlating image-sound pairs, the hit rate in a subsequent test, in which the model was able to find the desired target word from four images, was 61 percent. An AI model from OpenAI called CLIP, which also links images and text but, unlike CVCL, was trained with millions of image-text pairs, achieved 66 percent in the same test, just five percent more.
Where is the ball? A test for toddlers and AI models.
(Image: Wai Keen Vong)
When the images were linked to random utterances rather than the correlated ones (the child sees a ball, but the linked utterance is “cat”), CVCL’s hit rate dropped to 26 percent. This finding shows “the crucial role of consistent visual and verbal coexistence in learning,” the study says. In other words: In order to learn properly, both small children and AI models must be provided with the most accurate information possible.
Research into early childhood language acquisition
The model was also able to apply its learned knowledge to images that were not included in the training data. In this case, the model’s hit rate was on average 35 percent. For the researchers, this is proof that an AI model can effectively learn words and the associated objects in the world even with very limited data from a single child and does not necessarily have to be fed huge data sets.
At the same time, the study by Wai Keen Vong and his team also makes a contribution to research into early childhood language acquisition: “The successes of CVCL have an impact on theories of word learning,” they write. It is often assumed that classic word-picture learning is cognitively too difficult for small children, which is why other mechanisms are used. Despite some weaknesses, the study shows that “representation learning and associative, cross-situational mechanisms are sufficient to gain word-related associations from a child’s first-person experiences.”
(jl)
To home page