GPT or abandoned? AI future is a world-class model? Only three domestic AI models can survive?
Written by Cao Shuangtao
Editor/Yang Bocheng
Title picture/IC Photo
The popularity of Chat GPT has not yet dissipated, and a conference has pushed the attention of AI large models to a new level.
At the just-held Beijing Zhiyuan Conference, many legendary AI leaders were gathered, including Zhang Bo and Zhang Hongjiang, the most cutting-edge leaders in domestic AI, Geoffery Hinton, Yann LeCun, Yao Qizhi, and Joseph Sifakis. Winner of the Spirit Award, executives of AI companies such as Open AI founder Sam Altman, PaLM-E and RoBERTa.
Because each Zhiyuan Conference adheres to a professional academic ideological line, it has a high reputation among elite circles in the field of artificial intelligence at home and abroad, but to the public, it is a bit cold. At this conference, Sam Altman pointed out that in order to understand the development of general AI technology, Open AI must promote the reform of AI research.
But this statement has been opposed by many AI bigwigs. Among them, Stuart Russell, a professor at the University of California, Berkeley, criticized the Chat GPT and GPT-4 developed by Open AI for not “answering” questions, they do not understand the world, and they are not a developmental step in general AI. Yang Likun even directly pointed out that the current GPT autoregressive model lacks the ability of planning and reasoning, and the future GPT system may be abandoned.
In addition to fierce academic debates, how to supervise the current AI and the future development direction of AI have also become the focus of discussion at this conference.
01.How will follow-up AI be regulated?
Since 2023, while generative AI has swept many fields at a rapid speed, various problems caused by AI are also aggravating the concerns of the outside world.
In our country, “AI fraud” has become one of the directions of social concern recently. A few days ago, the police in Baotou, Inner Mongolia reported a case of using AI to carry out fraud. Mr. Guo, the legal representative of a company in Fuzhou, was defrauded of 4.3 million yuan within 10 minutes. According to reports, scammers use AI face-changing and onomatopoeia technology to pretend to be acquaintances to carry out fraud. Coincidentally, Xiao Liu from Changzhou, Jiangsu was sent voice calls and made video calls by a scammer pretending to be his classmate. After seeing the “real person”, Xiao Liu believed it was true and “borrowed” 6,000 yuan to the scammer.
Source: Douyin
In fact, behind the occurrence of AI fraud is still related to the rapid development of current AI technology and the continuous lowering of the threshold for technology synthesis. From the follow-up point of view, if the AI large-scale model technology continues to break through, it will gradually shift from facial synthesis to full-body synthesis and 3D synthesis technology in the future, and the synthesis effect will be more realistic.
In the United States, whether AI will affect elections has become the focus of local media discussions. According to the Associated Press, today’s sophisticated generative AI tools can “clone” someone’s voice and image in seconds, creating a large number of “fake materials.” As long as it is bound to a powerful social media algorithm, AI can quickly target audiences for dissemination, disrupting elections on an unprecedented scale and speed.
Other media in the United States predict that since the next presidential election in the United States will be held next year, it is not ruled out that the two parties in the United States will use AI technology for propaganda, fundraising and other activities. More importantly, because Chat GPT has excellent performance in text capabilities. As a result, a candidate’s team can generate a beautifully worded speech in seconds.
Based on various concerns about AI from the outside world, a few days ago, “AI Godfather” Jeffrey Hinton, Anthropic CEO Dario Amodai, Google Deep Mind CEO Demis Hassabis, etc. More than 350 AI executives and experts have signed a joint statement that “mitigating the risk of extinction posed by AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Regarding the question of how to supervise AI in the future, Sam Altman pointed out at the Zhiyuan Conference that Open AI is currently solving this problem in a variety of ways. First of all, as early as May 26, Open AI launched an incentive plan to invest 1 million US dollars to solicit effective AI governance solutions from the society.
Secondly, Sam Altman believes that humans are unable to detect some evil things in some malicious models. Open AI is currently investing in some new and complementary directions, hoping to achieve breakthroughs. But scalable supervision is an attempt to use AI systems to assist humans in discovering other system flaws, and explainability is to explain GPT-2 neurons with GPT-4. Although there is still a long way to go, Open AI believes in machine learning technology AI explainability can be further improved. At the same time, Sam Altman also believes that only by making the model smarter and more helpful in the future can we better realize the target advantages of general AI and reduce AI risks.
Finally, although Open AI will not launch the GPT-5 version in the short term, the world may have a more powerful AI system in the next ten years, and the world needs to prepare in advance. The core work of Open AI’s follow-up AI large model is still training, and it is preparing to establish a set of databases around the world to reflect the values and preferences of global AI, and share AI security research with the world in real time.
In addition to Open AI’s own efforts, Sam Altman also called for global efforts to improve the regulation of AI. For example, Sam Altman pointed out that China currently has some of the best AI talents in the world, considering that solving the difficulty of aligning AI systems requires the best minds from all over the world.
Therefore, Sam Altman also hopes that Chinese AI researchers can contribute to AI risks in the future. Tegmark also believes that China has done the most in the regulation of artificial intelligence, followed by Europe, and finally the United States.
Source: Zhiyuan Conference
In addition, Sam Altman also pointed out that there are difficulties in the cooperation of global AI regulation, but this is actually an opportunity. While AI is bringing the world together, a systematic framework and safety standards need to be introduced in the future.
However, considering the current intensification of the game between major powers in the world, the outbreak of multiple geopolitical conflicts, and the different attitudes of governments towards generative AI, this will make it difficult for global cooperation on AI regulation to be implemented in a short time. Affecting the market business of generative AI companies.
Europe has been at the forefront of AI regulation. In May, the European Union was close to passing a piece of legislation on artificial intelligence technology regulation. This is also expected to become the world‘s first comprehensive artificial intelligence bill and may become a precedent for developed economies.
European Commission President Ursula von der Leyen previously said in an interview with the media, “We hope that artificial intelligence systems are accurate, reliable, safe and non-discriminatory, regardless of their source. The introduction of relevant EU laws and regulations may make Open AI will subsequently withdraw from the EU market. Therefore, how to continuously improve its own generative AI model according to the adjustment of global regulatory policies is not only a problem encountered by Open AI itself, but also a problem that the entire industry needs to pay attention to.
02.Where is the future direction of generative AI?
It is undeniable that the current GPT-4 has been greatly improved in many capabilities. Compared with GPT-3.5, GPT-4’s performance in complex professional fields has been greatly improved, and its logical reasoning ability is also stronger. In the U.S. bar exam test, GPT-4’s score can reach the top 10%, but GPT- 3.5 can only reach the level of the last 10%.
Source: Open AI
The substantial improvement in capabilities has also allowed Chat GPT to open up more usage scenarios. At present, the Open AI official has also given several major application scenarios, such as adding AI to Duolingo for daily chat with users to accelerate users’ language learning; Morgan Stanley uses GPT-4 to manage its knowledge base to help employees Quickly access what you want.
However, many bigwigs have doubts about the current capabilities of GPT. Stuart Russell pointed out in his speech that Chat GPT and GPT-4 do not understand the world, nor are they “answering” questions. The current large language model is just a piece of the puzzle. What is missing from this puzzle and what will eventually be assembled , these are uncertain. The lack of many capabilities also determines that there is still a long way to go in the development of general artificial intelligence. Based on various doubts about the capabilities of GPT-4, Stuart Russell was also modifying the PPT during Sam Altman’s speech.
Source: Zhiyuan Conference
Yang Likun, winner of the Turing Award, one of the “Big Three in Deep Learning”, and Meta’s chief artificial intelligence scientist, holds the same view as Stuart Russell. He believes that the current autoregressive model of GPT lacks planning, which leads to its reasoning ability. At present, the whole is not working. If the large language model of autoregressive generation is purely based on probability, it is essentially impossible to solve the problem of hallucinations and errors. When the input text increases, the probability of errors will also increase exponentially.
In fact, the accusations of the two bigwigs against GPT are not unreasonable. Because the RLHF algorithm used by Chat GPT itself uses human perception to let the model judge the quality of its own answers and train itself to gradually give higher-quality answers. If you want to improve the reasoning ability of the model, you need to continuously iterate the algorithm while supplementing a large number of parameters in the database.
Source: Southwest Securities
However, the existence of various risks also prevents many generative AI companies from trying easily. If generative AI can achieve the same story reasoning ability and character emotion creation ability as novel writers, will this make generative AI completely out of human control? While this has caused global panic, will it encounter strong supervision from the local government, which will make the previous investment in generative AI go to waste?
Regarding the future development direction of generative AI, Yang Likun’s answer is the world model. This world model is not only a model that imitates the human brain at the neural level, but also fully fits the world model of the human brain in terms of cognitive modules. The biggest difference between it and the big language model is that it can have planning and prediction capabilities (world model) and costing capabilities (costing modules).
With the help of the world model, we can better understand the world and predict and plan the future. Through the cost accounting module, combined with a simple requirement (the future must be planned according to the logic of the most economical action cost), it can eliminate all potential poisons and unhealthy reliability.
Source: Zhiyuan Conference
But the problem is the parameters, algorithms, costs, etc. of the world model during training. Yang Likun simply gave some strategic ideas. For example, using a self-supervised model to train and establishing a multi-level thinking mode, etc., but Yang Likun could not give a complete plan for how to implement it.
Other participants did not share their views on the future development direction of generative AI. Therefore, the follow-up generative AI will still maintain the situation of each “doing its own thing”, and the globally unified generative AI may only stay in the laboratory stage.
03.Domestic Generative AI Prediction
Professor Huang Tiejun, dean of Zhiyuan Research Institute, said in an interview with the media after the meeting that the current problem with large generative AI models in China is that the industry is overheated, but the training data is too small, and the capabilities of tens of billions of models have just emerged. Although there are some technical capabilities in the middle, due to repeated efforts, this makes industry resources more dispersed, and at the same time, there is still a certain gap between its intelligence level and foreign generative AI large models.
As Professor Huang Tiejun said, taking Alibaba’s “Tongyi Qianwen” large-scale model as an example, because the training data of this large-scale model is a large number of Chinese dialogues and For text data, as well as text data from other sources, the pre-training data volume is about 200 billion words, which is equivalent to 14TB of text data.
The training data volume of Chat GPT is about 4.5 billion words, which is equivalent to 300GB of text data. The relatively small training data makes Ali’s “Tongyi Qianwen” also lacks multi-modal capabilities. In terms of text, there is a big gap between the two and GPT-4.
According to the “Large Language Model Comprehensive Ability Evaluation Report 2023” data released by the InfoQ Research Center, Chat GPT currently leads other domestic large model manufacturers with a comprehensive score of 77.13%.
Source: “Large Language Model Comprehensive Ability Evaluation Report 2023”
At the same time, Professor Huang Tiejun also pointed out that today’s large-scale models are an intermediate product of technological iteration. With the subsequent development of the domestic large-scale model industry, the ecologically reasonable number of large-scale models that can survive in the future is about 3.
As Professor Huang Tiejun said, Ma Huateng pointed out at Tencent’s internal high-level meeting that the dividends in the C-end market will disappear in the next ten years. The whole hope is in the ToB end market, and the second half of the Internet belongs to the industrial Internet. Ali Business Research Institute also previously pointed out that the next ten years will be a golden outlet for the transformation of traditional enterprises.
But from the perspective of the ToB end market, taking the SaaS market that has been developed for many years as a reference, if the large model wants to truly open the ToB end market, its core must be to bring the value of “cost reduction and efficiency increase” to customers, especially in the current situation. This is especially the case when few industries are still on the sidelines for big models. The traditional manufacturing industry is the mainstay. At present, the common problem encountered by small and medium-sized traditional manufacturing industries is the decrease in orders. The industry has been fighting in the price war, and the downstream payment cycle has become longer. Many small and medium-sized manufacturing industries are currently struggling to support. In order to avoid high trial and error costs, many small and medium-sized manufacturing companies naturally dare not easily try the use of large models.
And from the perspective of the development history of the SaaS industry, the domestic SaaS industry experienced a wave of growth in 2015 after 10 years of calm since the beginning of 2004. Since the outbreak of the epidemic in 2020, the epidemic has accelerated the digital transformation of enterprises, and the domestic SaaS market has entered a critical growth period. But even so, the current domestic SaaS industry ecology is not yet complete, and the market is not yet mature.
Source: Flash Cloud
Obviously, it is not overnight for large models to open up the TOB market, but an extremely slow process. Moreover, the costs incurred due to algorithms, computing power, and data during model iteration, including the subsequent launch of various functions, require large model companies to continuously invest large amounts of money.
Commercialization takes a long time to implement, high capital investment, and the existence of problems such as difficulty in making profits in the short term. In the future, companies lacking cash flow will have to shut down their own large-scale models in the face of financial pressure, and industry resources will become more and more expensive. Concentrate on the head model maker.
However, judging from the experience of multiple industries such as online car-hailing and food delivery, in an emerging industry after years of turmoil, there are only about three companies that can really develop in the future, and many other companies have been submerged in history. in the long river.
04.epilogue
It is undeniable that generative AI capabilities are gradually being recognized by the market, but how to regulate AI will be an issue that many countries around the world will continue to pay attention to.
As for us ordinary people who are worried about whether AI will take away their jobs in the future and cause their own unemployment, perhaps as Tegmark said, the economy and the job market will change faster and faster. If you are strong in basic knowledge, And being very creative and open minded, you have the flexibility to go with the flow.