Last week, Huida and several Chinese scientific research institutions published papers with similar contents. First of all, Huida developed a new game AI agent (Agent) ── “Voyager”, which connected the large AI model GPT-4 to the sandbox game “Minecraft”.
Based on GPT-4, Voyager can expand items and equipment in
（Source：Traveling）
Then, Shangtang Technology, Tsinghua University, Shanghai Artificial Intelligence Laboratory and other institutions announced the general AI intelligence “Ghost in the Minecraft” (GITM).
Also integrated the Large Language Model (LLM) into
▲ GITM’s task coverage rate of “Being a God of Creation” is much higher than that of existing AI. (Source: Github)
To put it bluntly, both sides chose to use “Being a God of Creation” to train the AI, and the AI has completed more tasks than real players, which is comparable to the real one (it was impossible a few months ago). Putting aside the seemingly difficult technical details, why did both sides use “Being a God of Creation” as an experiment?
“Being a God of Creation” is a large-scale AI experimental field
Speaking of the AI trained with
Because of this situation, when AI was not as developed as it is now a few years ago, scientists initially only used “Be a Creator” to train AI to complete simple instructions. In 2019, Facebook developed the “CraftAssist bot” AI assistant “CraftAssist Bot”, which is a smart assistant robot that performs various tasks specified by players, such as taming horses, building cities, meeting with villagers and trading.
Also in 2019, Carnegie Mellon University, Microsoft, DeepMind, and OpenAI, together with NeurIPS, the top machine learning conference, held the “MineRL” AI competition dedicated to “Being a God of Creation”. It will hold four sessions by 2022. Witness the process of AI applications from simple to complex.
The first MineRL was simple: see which team can create an AI that successfully mines diamonds. But unfortunately, in the end, none of the more than 900 top teams completed the challenge. Some AI can chop wood but can’t synthesize it. Some AI can make a furnace but can’t smelt it. Dig down. The content of the third competition was to see who could train an efficient AI. The champion was Tencent AI Lab “Jue Wu” who won the championship with a score of 76.970.
In fact, it has been nearly five years since major companies began intensively training AI with “Be a God of Creation”. The U.S. Defense Advanced Research Projects Agency (DARPA) launched the Adaptive Distributed Probabilistic Task Assignment (ADAPT) program in August 2020, with a view to using AI on the battlefield to assist the commander in making decisions. The contractor Aptima used the “Being a Creator” training AI interacts with humans.
In 2021, researchers from Copenhagen University of Information Technology, New York University, and Shanghai University also created a 3D Neural Cellular Automata (NCA) system and put it into “Being a Creator”. At this time, NCA can already build caterpillars and castles , apartments and trees and other complex objects.
The more well-known experiment was last year. OpenAI will announce in 2022 that AI is already proficient in playing “Being a Creator”, and introduces a novel and convenient AI training method “Video PreTraining” (Video PreTraining). After giving the AI 70,000 hours, the AI is quite proficient in operation and has mastered high-level gameplay that humans can only comprehend.
DeepMind also developed an AI comparable to OpenAI’s achievements earlier this year, called “DreamerV3”. DreamerV3 did not need any human data to assist, threw it into “Being a God of Creation” and worked hard for 17 days. It amazingly learned how to dig diamonds from scratch, and became the world’s first AI that can quickly mine diamonds by exploring by itself.
Introducing DreamerV3: the first general algorithm to collect diamonds in Minecraft from scratch – solving an important challenge in AI. 💎
It learns to master many domains without tuning, making reinforcement learning broadly applicable.
Find out more: https://t.co/7GP0R93Yvc pic.twitter.com/YacIDojAF1
— Google DeepMind (@DeepMind) January 11, 2023
William Guss, one of the initiators of the MineRL competition, couldn’t help sending a congratulatory message immediately: After 4 years, the “Diamond Challenge” has finally been overcome!
There is no more suitable laboratory than “Being a God of Creation”
“Be a God of Creation” has been out for 12 years. It is one of the most popular games in the world. Many players describe it as an electronic version of Lego. In 2019, it surpassed “Tetris” to become the world’s highest-selling game. The “open world” attribute makes it an excellent place for scientists to train AI.
Unlike “League of Legends” and other single strategy battle games, “Be a Creator” does not have an ultimate goal, and the only two modes are “survival” and “creation”. In “Survival” mode, players need to build houses and collect items. The longer they live, the better. Even the sword that repels zombies and spiders requires players to forge with wood and iron ore; “Creation” mode is more free, and players can build houses with cubes of various materials and tool materials, and the game is never over.
If it is said that “League of Legends” or board games allow AI to learn and surpass humans at an extremely fast speed due to simple rules and mechanisms, the complexity of “Being a God of Creation” almost challenges the limits of AI. Diverse environments, maps generated entirely by random seeds, long decision sequences and complex skill learning, high freedom of play and other massive strategic preferences all increase the difficulty of AI research.
For AI to find diamonds within 15 minutes, it needs to collect logs, synthesize wood planks, wooden sticks and wooden picks with bare hands, collect iron ore, and undergo a series of processing to synthesize diamonds. It seems like a trivial matter, but it needs to be mastered by AI. Very complicated logical relationship──This is only a small part of “Being a God of Creation”. In addition to making tools, the AI needs to set its own goals and learn to explore the game until all tasks are unlocked.
▲ “Being a God of Creation” Diamond digging steps.
If it’s just learning to play chess or League of Legends, the AI just needs to know how to win. Do you think that the AI trained in “Being a God of Creation” is much more advanced? (Although the learning rate may be slower)
At present, in AI research, scientists are increasingly pursuing general artificial intelligence (breaking latest news) to create a more general AI, that is, AI can master a wide range of skills, adapt to various environments, and deeply simulate human ability to solve complex problems, such as human perception, understanding and interaction. , rather than a “small fight” AI that can only answer simple questions and beat humans in chess. This means that to train a general-purpose AI, the environment of “Being a God of Creation” is very suitable.
But there are so many open sandbox games, why choose “Being a God of Creation”? Some netizens asked Jim Fan, the chief scientist of Huida who developed Voyager, since Voyager was born for the open world, why didn’t he choose to train AI with games such as the “Legend of Zelda” series or “Elden Circle”?
Jim Fan replied that the main difficulty is not the algorithm, but because applications such as The Legend of Zelda and Elden’s Ring are “slow and difficult to programmatically control high-end consoles” and “requires a good program API to control characters “. And “Being a God of Creation” has a lower technical threshold, and the goal and gameplay are similar, which actually reduces the difficulty of training AI.
I’ve been dreaming about these for a while. The main difficulty is not the algorithm, but:
– Simulator: slow and difficult to programmatically control high-end consoles.
– API: needs a good coding API to control the character. Pixel -> action is way too hard in Zelda & Elden… https://t.co/aEZ0hwNpZ6— Jim Fan (@DrJimFan) May 29, 2023
And “Being a God of Creation” is also very friendly to developers. Many libraries that build “Being a God of Creation” are open source, so people can easily modify them. It also allows researchers to record and learn how players complete tasks and observe how they interact with AI. cooperate. From these aspects, there is no game more suitable for training AI than “Being a God of Creation”.
What’s so special about the new AI of “Being a God of Creation” this time?
The biggest difference between Huida Voyager, SenseTime and Tsinghua University and Shanghai Artificial Intelligence Laboratory’s GITM, compared with the past AI training based on “Being a God of Creation”, is that it introduces a large language model (LLM) as the training core. In the past, the mainstream methods of training AI in “Being a Creator” were imitation learning and reinforcement learning. Even the most advanced AI developed by OpenAI and DeepMind is trained by imitation learning and reinforcement learning.
If you want AI to learn mining, what you need to do in simulation learning is to feed AI tens of thousands of game videos, let it analyze and learn why trees are cut down, how to use the workbench to decompose and synthesize them, how to make pickaxes, how to smelt iron ingots, Making iron pickaxes, digging diamonds, how to avoid falling into magma, how to fight and other necessary steps. After the feeding and learning is over, the AI will then undergo intensive learning, that is, throwing the AI into the game to “really” mine, discover and solve problems through repeated experiments, and learn mining skills.
But with the large-scale language model training method, the AI learning logic is completely reconstructed. Using mining as an example, AI first asks questions based on the overall goal of “discovering as many materials as possible to help mining”, and automatically generates many small tasks.storeIf there is a program that helps to successfully solve a certain task (for example, it is correct to cut down trees first and then synthesize them, and it is correct to smelt them to make a furnace), AI will gradually build up the correct mining skill library. When encountering a mining task later, the AI can go to the skill library to search for the task steps according to the description.
▲ GITM of large language model training method. (Source: Github)
That is to say, the AI of the large-scale language model training method can automatically explore and master skills, making it more able to help AI become a lifelong learner, gradually acquire, update, accumulate and transfer knowledge for a long time, and at the same time alleviate the “disaster” of other continuous learning methods. sexual oblivion”. It’s a bit like the difference between letting AI stay in school all the time, or asking AI to explore society from the very beginning. All in all, the AI trained by the large language model method is more like a real person.
▲ Voyager explores more items than many AIs. (Source: Voyager)
An ideal AI should have abilities similar to human beings. In “Being a God of Creation”, AI proposes suitable tasks regardless of skill level and world status (if you find yourself in a desert instead of a forest, you will learn to obtain sand and cacti before forging iron), and according to the environment Feedback to improve skills and remember learned skills (such as the skills of fighting zombies and spiders), or constantly exploring the world, automatically looking for new tasks, are very close to the real human decision-making process and behavior.
It is not an exaggeration to say that “Be a God of Creation” is a reduced version of the real world. What is frightening is that with the publication of Voyager and GITM research results, AI has been able to break through “Be a God of Creation”.
People can’t help but think, will AI soon be able to be like a fish in water in the real world? It can live like a human, know how to cook, use a mobile phone, know how to wear clothes, how to keep pets, how to take public transportation… What’s even more incredible is that it can also have ideas like a human, knowing which clothes look good, How to cook better, who to call… Maybe in the near future, the “people” we meet will no longer be able to tell whether they are AI or not.
Now that they can break through “Being a God of Creation”, it is not impossible for AI to live a “human life” technically. And how will humans choose?
(This article is reprinted with authorization from Pinwan; source of the first picture: Flickr/downloadsource.fr CC BY 2.0)