Author | Li Mei
Editor | Chen Caixian
One day in 1764, the “Spinning Jenny” was born in the home of a British weaver Hargreaves, which increased the weaving efficiency by eight times and sounded the clarion call for mankind to enter the machine age. Soon, this “wind of machines” spread to various fields such as coal mining, metallurgy, manufacturing, and transportation.
A technology can extend its tentacles to all walks of life, relying on its underlying versatility. As Baidu CTO Wang Haifeng said, deep learning has strong versatility, showing the characteristics of standardized, automated and modular industrial mass production. In the fourth industrial revolution centered on artificial intelligence, it is the versatility of deep learning technology that opens up the space for AI to be applied on a large scale. Among them, the deep learning framework platform and the large model assume the role of the technical base and are entrusted with the heavy responsibility of the infrastructure in the AI era. The former is the hub for efficient execution of various neural networks under different hardware conditions, while the latter uses pre-training The technology adapts to various upstream tasks.
Entering 2023, the next golden decade of deep learning will drive towards the era of AI industrial mass production. How do we go through the fog and find the best path for AI technology innovation and industrial landing? Baidu, which has more than ten years of technical accumulation in the field of deep learning, promptly gave an answer: “Deep Learning +”.
At today’s Baidu Create AI developer conference, Baidu CTO Wang Haifeng delivered a keynote speech entitled “”Deep Learning +”, a new engine for innovation and development”. He proposed:The technological innovation and industrial development of artificial intelligence has entered the “deep learning +” stage.
Wang Haifeng at the Baidu Create AI developer conference
Wang Haifeng revealed the connotation of “+” in “deep learning+” at three levels:
From a technical point of view, yesdeep learning + knowledge;
Ecological point of view, yesDeep learning + upstream and downstream ecological partners;
From an industry perspective, yesDeep learning + thousands of industries.
For AI to move towards large-scale production, these three levels are indispensable.
01
Knowledge: AI is a generalist and an industry specialist
In the past few years, deep learning models have achieved great success in various fields such as vision, NLP, and speech. However, a general consensus in the industry is that the current model still has the disadvantages of poor explainability and poor versatility, and the performance of the model is still not good enough. There is a lot of room for improvement.
A key reason is that the model lacks prior knowledge input during the learning process. This has built a wall technically, blocking the way for AI to move towards large-scale industrialization.
In Wang Haifeng’s view, deep learning + knowledge is an important direction for the further development of AI technology.
Human reasoning ability depends on knowledge, knowledge condenses the wisdom of human beings for thousands of years, captures and recognizes domain knowledge and encodes it into the model, and improves the memory and reasoning ability of the model for knowledge. This knowledge-enhanced deep learning paradigm is even more to approach the human brain.
In the era of large-scale models of AI, injecting more wisdom into intelligence is exactly a distinctive large-scale model route that Baidu has taken.
In 2018, Google released the language model BERT, which set off a technical revolution in the pre-training model in the NLP field. Baidu is the first group of institutions to occupy high ground in China. In March 2019, Baidu released and open sourced Wenxin ERNIE 1.0, the first knowledge-enhanced language model in China. The performance exceeds BERT. By introducing knowledge graphs and integrating massive data with rich knowledge from multiple sources, the learning efficiency of the model is greatly improved, and the interpretability is also enhanced.
Behind this first step is Baidu’s deep technical accumulation in the NLP field. As early as 2010, Wang Haifeng, a top international NLP expert, joined Baidu and led the establishment of the first NLP R&D department in China. Now he is also working on large models. earliest.
Today, the maximum parameter scale of Wenxin series of large models has evolved to the level of 100 billion, and many technological breakthroughs have been made in the fields of language understanding, text generation, and cross-modal semantic understanding. Following the route of “deep learning + knowledge”, Baidu has embarked on a differentiated road of large models, and has gone very steadily and for a long time.
The development history of Baidu Wenxin large model
Looking at the development of various large models in China, compared with other players, Baidu has withdrawn from the blind wave of parameter competition earlier and established aAnother “killer feature” of Wenxin: industrial grade.
Behind this is still the logic of “deep learning + knowledge”: in the process of AI moving towards large-scale industrialization, while the large model has general knowledge, it must also “learn from teachers” from all walks of life, and then become proficient “Experts” in various fields.
Therefore, “+knowledge” adds not only scientific knowledge and empirical knowledge, but also industry knowledge.
In order to fill the gap between the basic model and application scenarios, on the basis of Wenxin’s general-purpose large model, Baidu cooperates with leading enterprises and institutions in various fields, and uses industry knowledge enhancement technology to integrate the unique data and knowledge of each industry for the first time. Integrating into the large model, the capabilities of the large model are adapted and extended to many fields such as energy and power, finance, aerospace, media, film and television, automobiles, urban management, gas, insurance, electronic manufacturing, and social sciences.
Baidu’s 11 large-scale industry models outline the prototype of the future AI industrial production landscape.
Baidu Wenxin industry model
At present, the industrialization of large models still has a long way to go, and the popularity of the AIGC track in 2022 has pointed us to the clearest path for large models to be implemented.
The Vincent graph model DALL·E 2 released by OpenAI in April last year fired the first shot of the AI painting boom, and then Stability AI launched the open source Stable Diffusion model in August, which completely boiled the circle of AI painting.
On August 19, three days before the launch of Stable Diffusion, Baidu released an AI painting product – Wenxin Yige. It is based on Wenxin large-scale model technology, and can generate high-definition paintings of various styles based on text. It is a “magic brush” for ordinary people to create high-quality art. In the long run, AI painting has a broader scene on the toB side. After solving the problems of copyright and generation controllability, tools like Wenxinyige can be used in scenarios such as mobile content production, games, and industrial design. will have wide application.
Wenxinyige official website
The huge imagination space of AIGC is rooted in the cross-modal ability of large models. Multimodal learning is the key direction of deep learning in the next decade. Just as humans can conceive a picture from a text description, the model can have the ability to generate cross-modality after integrating multimodal knowledge such as language and vision. , Wensheng diagrams and Wensheng videos will be the outlets for large models in the future.
This is also the extension of the concept of “deep learning + knowledge” at the modality level.
02
Ecology: take a flying paddle to sail deeper into the industry
A technology that lacks an ecosystem will eventually be eliminated by history.
For AI technology to move towards industrial mass production, deep learning frameworks, basic algorithms, AI chips, data, applications, and talents are all indispensable. In the AI industry chain, every demand and feedback needs to be smoothly transmitted to every link of deep learning technology and application. Only through continuous iterative optimization of each link can the technological innovation and industrialization of AI be accelerated.
It is based on this kind of thinking that Baidu put forward the viewpoint of “deep learning + upstream and downstream ecological partners”.
In such an ecosystem, the deep learning framework bears on the neural network model and application, and connects to various chips, so it is in a very core position. Comparing the operating system Windows in the PC era, and the operating systems IOS and Android in the mobile era, Wang Haifeng believes that the deep learning framework can be said to be the “operating system in the smart era” today.
A good framework can help developers and enterprises greatly improve the efficiency and effectiveness of deep learning model development and avoid repeated “wheel-making”. Especially for large models, the framework can solve difficult problems such as low development efficiency, slow reasoning speed, uncontrollable deployment costs, and difficult chip adaptation.
Baidu’s paddle paddle platform (PaddlePaddle) serves as a technical base, and its powerful capabilities have now been verified on the large models of the Wenxin series. Looking back at the birth of the flying paddle, from the very beginning, it was not only an achievement of Baidu itself, but also a major leap in the history of China’s AI development. As early as 2012, Baidu began to explore deep learning technology and applications. In January 2013, Baidu established the world‘s first Institute of Deep Learning (IDL), and began to develop a deep learning framework. In 2016, it finally open sourced the first domestic deep learning framework PaddlePaddle. In the domestic market, PaddlePaddle is the only deep learning framework that can compete head-on with TensorFlow and PyTorch, two major international frameworks.
Looking closely at the research history of the Chinese in the field of deep learning frameworks, we will find such an evolution: the early representative frameworks such as Caffe (2014) and MXnet (2015) all started in academia and came from a group of PhD students abroad. The hands of Chinese students, and starting from PaddlePaddle, Baidu has injected deep industrial genes into the deep learning framework. It was not until 2020 that various companies launched their own frameworks one after another, and the domestic framework ushered in the first year of explosion in the industry.
In 2019, PaddlePaddle has the Chinese name “Flying Paddle”, which literally means “very fast boat”. Today, on this fast ship, AI has sailed to the deep water area where the technology has landed. The Flying Paddle platform has gathered 5.35 million developers, created 670,000 AI models, and served 200,000 enterprises and institutions.
Baidu flying paddle panorama
It can be seen that more and more enterprises are carrying out low-threshold AI application and development with the help of Paddle’s AI technology ecology. For example, small and medium-sized enterprises can develop a large number of different types of technical service applications or models based on Flying Paddle, while large enterprises can use Flying Paddle to improve business operation efficiency by virtue of their own data advantages. Based on the core framework used for R&D models, model libraries containing various trained deep learning models, development kits and tool components that support low-code, as well as the zero-threshold AI development platform EasyDL and the full-featured AI development platform BML of Flying Paddle Enterprise Edition , Flying Paddle has opened up the capabilities of AI to all walks of life.
In fact, from an ecological point of view,Baidu is not only the only one in China, but also one of the very few artificial intelligence companies with a full-stack layout in the world.
At the computing power level of AI, Paddle has also built a highly competitive hardware ecosystem. Flying Paddle is closely cooperating with domestic and foreign hardware manufacturers to carry out joint optimization of software and hardware. Up to now, more than 30 hardware manufacturers have deeply integrated and optimized Flying Paddle, and mainstream chips at home and abroad have basically been adapted to Flying Paddle. In 2022, Flying Paddle also released the “Hardware Ecological Co-creation Plan” with its hardware partners. At present, the number of partners has reached 28, including hardware manufacturers such as Intel and Nvidia.
From core technologies such as China’s first cloud-based full-featured AI chip “Kunlun”, Flying Paddle deep learning framework, Wenxin large model, to applications such as search, smart cloud, autonomous driving, and smart home, Baidu’s own technological innovation and implementation system , is a microcosm of domestic AI moving towards industrial mass production.
03
Thousands of industries: Baidu’s big vision
Where will AI technology end up? In Baidu’s view, there must be thousands of industries.
Relying on the Flying Paddle platform and the Wenxin model, Baidu has not only achieved large-scale applications in its own search, information flow and other businesses, but also set its sights on a wider range of industries: industry, agriculture, energy, cities, science, etc. Computing and more than 20 fields.
Applying deep learning technology to reduce costs and increase efficiency has basically become a consensus in all walks of life. However, enterprises must actually apply AI to solve practical problems and face many difficulties:
First of all, each industry has its own problems. Although the number of industries can be counted, the number of subdivided scenarios is difficult to count. It is difficult for AI companies that provide algorithms and models to understand the special scenario needs of each industry. Secondly, most traditional industries do not have their own AI R&D teams, and the cost of algorithm production is relatively high, which not all companies can afford. In addition, there is no deployment hardware system suitable for industrial scenarios, which has also become the “last mile” problem for AI to land.
To this end, the solution given by Baidu Flying Paddle is to provide enterprises with the ability equivalent to “an entire algorithm team” based on the characteristic data of the industry.
Take the example of manufacturing. Before each aircraft takes off and after landing, aircraft line maintenance personnel need to carry out comprehensive inspections, but it is difficult for manual inspections to be 100% error-free. Therefore, many aircraft maintenance service companies hope to develop an AI application for auxiliary maintenance, but without a professional algorithm team and insufficient budget, it is impossible to start this matter. In Sichuan Safeway Aircraft Maintenance Service Co., Ltd., an IT project leader finally solved this problem with the help of Baidu Flying Paddle EasyDL platform. With no experience in AI algorithms, he developed a maintenance safety guard system based on the EasyDL platform, which can efficiently detect aircraft airspeed sleeves, flight records, etc., and has already implemented it at Changsha Huanghua Airport.
Aircraft pitot tube and tube sleeve detection by aircraft maintenance safety guard system
Flying paddles are also allowing AI to enter the fields. In Zhangziying Town, Daxing District, Beijing, Baidu Flying Paddle, BOE Houji and Yunong jointly built an AI smart plant factory. The gram weight recognition model built based on Baidu AI technology algorithm can recognize the weight and health status through the pictures taken, with an accuracy rate of over 95%; the target detection model based on the flying paddle EasyDL platform realizes automatic recognition of common insects, and the recognition accuracy reaches 90%.
Object detection model for early warning of pests
In the past, the only agricultural expert in the factory had to walk 20,000 to 30,000 steps a day in order to observe the growth of vegetables and insect damage. Now, in a space of 2,600 square meters, AI is taking care of the plants 24 hours a day, and the two workers only need to do Some basic work, the overall work efficiency is greatly improved. Smart plant factories like this are being implemented in smart greenhouses across the country.
There is a huge and rich industrial system in China, which brings great opportunities for the implementation of AI technology. Baidu Fei Paddle is making powerful AI available to everyone, providing answers to questions in every industry. At the same time, rich application scenarios will in turn promote the breakthrough of the underlying technology itself, forming a virtuous circle.
04
write at the end
David Mitchell said in his science fiction novel “Cloud Atlas”: History is a deck of cards, from which a few cards are drawn at random. Our ancestors drew 3, 4, 5, while our generation The ones drawn are 10, J and Q.
Each generation has its mission. In the first ten years of deep learning, the ability of AI has continuously broken the ceiling and refreshed human cognition in algorithms, academic papers, and paper projects; and in the next ten years, the industrial production of AI is the mission that falls on our shoulders. In this process, a large number of domestic manufacturers must take on the role of leaders. As far as Baidu is concerned, it has never missed every important node of deep learning in the past ten years. Today, Baidu is the first in the industry to propose the concept of “deep learning +”, which is another forerunner’s vision.
As the international competition in artificial intelligence becomes increasingly fierce, Baidu has already blazed a clear path for where China’s AI will go in the next decade. Taking large-scale models as an example, foreign technology giants such as Google, Microsoft, Meta, OpenAI, and Nvidia are all competing on this track. In contrast, in terms of resource investment, technological innovation, and commercialization, the domestic large-scale It is not optimistic. ChatGPT, which has been popular in the circle recently, has caused many domestic researchers to cry out that “this is the autumn of life and death”.
Under the vision of AI deepening the industry, the domestic large-scale model with domestic architecture is the core competitive advantage that distinguishes Baidu from other AI players. At the beginning of 2023, Baidu used “deep learning +” to initially create an AI industrial mass-production universe, looking forward to the arrival of more residents.
(Public account: Leifeng.com(Public number: Leifeng.com)）
Leifeng.com original articles, unauthorized reprinting is prohibited. See Reprint Instructions for details.
