Home » Artificial intelligence: model helps robots learn like humans

Artificial intelligence: model helps robots learn like humans

by admin
Artificial intelligence: model helps robots learn like humans

Artificial intelligence: model helps robots learn like humans

In the summer of 2021, OpenAI quietly shut down its robotics team, citing progress being hampered by a lack of appropriate data. Data that is necessary to train robots how to move and think with artificial intelligence (AI). In mid-March, three of the early OpenAI researchers announced that their start-up Covariant, founded in 2017, had solved this problem and presented a system that combines the thinking abilities of large language models with the physical dexterity of an advanced robot.


The new model, called RFM-1, was trained using years of data collected from the small fleet of Covariant robots that customers like Crate & Barrel and Bonprix use in warehouses around the world, as well as text and videos from the Internet. The model will be released to Covariant customers in the coming months. The company hopes that the system will become more powerful and efficient over time as it is put into practice.

So what can it do? At a demonstration in early March, Covariant co-founders Peter Chen and Pieter Abbeel showed how users can stimulate the model with five different types of input (prompts): text, images, video, robot instructions and measurements. For example, they showed the robot a picture of a container filled with sports equipment and told it to pick up the package with tennis balls. The robot can then grab the item, create an image of what the container will look like when the tennis balls are gone, or create a video showing what the robot will look like while completing the task, from a bird’s eye view.

See also  Sony's Alleged Plans for PS5 Slim and Project Q Revealed in Microsoft's Lawsuit

If the model predicts that he can’t grasp the object properly, he might even report back: “I can’t grasp the object properly. Do you have any tips?” One answer might advise him to use a certain number of suction cups on his arms for better grip, for example eight instead of six.

Chen said this is a major advance for robots that can adapt to their environment using training data, rather than the complex, task-specific code that powered the previous generation of industrial robots. It’s also a step toward workplaces where managers can give instructions in human language without worrying about the limitations of human labor: “Pack 600 meal kits of red pepper pasta using the following recipe. Don’t take breaks!”

According to Lerrel Pinto, who directs the General Robotics and Artificial Intelligence Laboratory at New York University and has no connection to Covariant, although robotics researchers have built simple multimodal robots and deployed them in laboratories before, large-scale deployment of a robot Being able to communicate in so many modes represents an impressive achievement for the company.

To beat its competitors, Covariant needs to collect enough data so that the robot can be used in the wild, says Pinto. He will be put to the test in warehouses and loading docks, constantly interacting with new instructions, people, objects and environments. “The groups that train good models will be those that either have access to already large amounts of robot data or are able to generate that data,” he says.

According to Covariant, the model has a “human-like” ability to think, but it also has its limitations. During a demonstration that featured a live broadcast of a Covariant robot and a chat window for communicating with it, Chen invited me to give the model a task. However, when I asked the robot to “return the banana to shopping bag two,” it struggled to retrace its steps, resulting in it picking up a sponge, then an apple, and then a variety of other items before finally completing the task solved with the banana. “It doesn’t understand the new concept,” Chen explained, “but it’s a good example – it might not work as well if you don’t have good training data.”

See also  YouTube's fight against spam: the news to block harassing comments

The company’s new model embodies a paradigm shift taking place in the world of robotics. Instead of teaching a robot how the world works through instructions like physics equations and codes, researchers teach it the same way humans learn: through millions of observations. The result “can really act as a very effective, flexible brain to solve any robotic task,” says Chen.

The playing field is likely to become even more crowded this year for companies using AI to power more nimble robotic systems. Earlier this month, humanoid robotics startup Figure AI announced a partnership with OpenAI, raising $675 million from tech giants like Nvidia and Microsoft. Marc Raibert, the founder of Boston Dynamics, recently launched an initiative to better integrate AI into robotics. This means that advances in machine learning will likely also lead to advances in robotics.

However, some questions remain unresolved. If large language models continue to be trained on millions of words without compensating the authors of those words, perhaps robotic models will also be trained on videos without paying their creators. And if language models hallucinate and perpetuate bias, what analogous problems will there be in robotics?

Covariant will push development forward for now, as RFM-1 is designed to continually learn and improve. Ultimately, the researchers want to train the robot with videos that the model creates itself – a type of meta-learning that causes many headaches and also raises the question of what happens if the errors made by the model accumulate. Unfortunately, with the current hunger for more training data, researchers see this as almost inevitable. “This training will be a reality,” says Abbeel. “When we talk about it again in six months, we will talk about this.”

See also  The amnesty on cryptocurrencies may be worth three billion. But it will be difficult to get them


To home page

You may also like

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More

Privacy & Cookies Policy