What is Baidu ERNIE Bot? What do you expect from ERNIE Bot?

Baidu ERNIE Bot is an ancient poem that is updated daily, aiming to stimulate readers' thinking and enhance humanistic spirit. I expect ERNIE Bot to bring me more beautiful literary experiences, deeper cultural artistic conception and more interesting historical stories. In mid-March, global technology giants once again competed to appear on the big language model track.

Within a week, OpenAI, an American startup that developed ChatGPT, Microsoft, a technology giant that invested heavily in OpenAI, and Baidu, a leading Internet company in China, released the latest progress in LLM. This has once again triggered global attention in this field.

On March 4th, local time, OpenAI released the latest version of its large-scale language model-GPT-4. Compared with GPT-3.5, the quality and technology of Q&A have been significantly improved.

On the afternoon of March 16, Baidu launched the ERNIE Bot test of a new generation of big language model and generative artificial intelligence products, thus becoming the first China enterprise to join the competition in this track.

At the press conference, Li Yanhong, founder, chairman and CEO of Baidu, showed five usage scenarios of ERNIE Bot's literary creation, business copywriting, mathematical calculation, Chinese understanding and multimodal generation through a question-and-answer session. A few hours later, Microsoft announced that it would connect GPT-4 to the whole Office bucket, and the new name was "Microsoft 365 copy".

Just like the article published in February 17 of Finance and Economics Law (OpenAI exclusive response | Why |ChatGPT is not open to all domestic users for registration? ), mobile phone numbers in Chinese mainland, China and Hongkong, China cannot be registered with ChatGPT account. In addition, although the application programming interface (API) of OpenAI has been opened to 16 1 countries and regions, it does not include China, Chinese mainland and China.

On the one hand, the industry is generally concerned, who will be the next wave of technology in the overwhelming wave of AIGC (Generative Artificial Intelligence)? On the other hand, in the sensitive period of technology competition between China and the United States, all parties are also concerned about the ripple caused by Baidu's first step, and how China enterprises should respond.

0 1 "Are you really ready?" On March 16, Li Yanhong gave a speech in a white shirt and sneakers. At first, I faced the problem directly. "Recently, many friends asked me, why today? Are you really ready? " ?

Li Yanhong's answer is that although Baidu has invested more than ten years in AI research and made full preparations for the release of ERNIE Bot, it can't be said that it is completely ready, because ERNIE Bot has a high benchmark test threshold for ChatGPT and even GPT-4, and there are "many imperfections". However, he stressed that "once there is real human feedback, ERNIE Bot will make great progress".

Li Yanhong explained that the reason why he chose to publish on the same day was because there was demand in the market: customers and partners wanted to use the latest and most advanced big language model earlier.

How to understand what Li Yanhong said "GPT-4 benchmark test threshold is very high"?

On March 4th, local time, OpenAI released the latest version of its large-scale language model-GPT-4. It is worth noting that GPT-4 is a large-scale multi-modal model, that is, it can accept input of images and text types. GPT-3.5 can only accept text input.

In the demo video, Greg Brockman, president and co-founder of OpenAI, drew a sketch of the website with pen and paper and entered the picture into GPT-4. After only 1 to 2 seconds, GPT 4 generated the web page code and made a website highly similar to the sketch. According to the experimental data published by OpenAI, GPT-4 model has made great progress compared with the previous generation GPT-3.5, and has surpassed the level of most human beings in many professional tests.

Pan Helin, co-director of the Digital Economy and Financial Innovation Research Center of Zhejiang University International Joint Business School, believes that ERNIE Bot needs to be fully open to users in the future. Whether it is through the B-side API or directly open to C-side users, user experience word of mouth is the last word. At present, ChatGPT is not open to users in China. In the domestic market, Baidu will have the first advantage.

Zhang Yi, CEO and chief analyst of Ai Media Consulting, who has evaluated the products of OpenAI and Baidu, said that GPT series models including GPT-4 and ERNIE Bot are essentially the same kind of products, but their data coverage areas and data model accumulation lengths are different. In the short term, OpenAI's product preparation time is relatively sufficient, and intelligence is temporarily ahead. But for ERNIE Bot, it is unusual to cultivate such a product in such a short time.

At the same time, Zhang Yi also has more confidence in Baidu to make better products. His reason is that China will have more advantages in the talent pool of artificial intelligence, big data and big models.

Chen Duan, director of the Research Center for Digital Economy Integration, Innovation and Development of the Central University of Finance and Economics, believes that compared with overseas competitors, Baidu's greatest advantage is that it has built a moat of understanding in language and culture.

As a large-scale language model product developed by China Company, ERNIE Bot's Chinese comprehension ability has attracted much attention. The important reason is that many commentators think that ChatGPT's Chinese question-and-answer ability is not as good as English.

Li Yanhong said that as a big language model rooted in the China market, ERNIE Bot has the most advanced natural language processing capability in the Chinese field. In the live exhibition, ERNIE Bot correctly explained the meaning of the idiom "Luoyang Paper Expensive" and the corresponding economic theory, and also wrote a Tibetan poem with "Luoyang Paper Expensive".

Li Yanhong said that ERNIE Bot's training data includes: trillions of web pages, billions of search data and picture data, tens of billions of daily voice calls and 550 billion factual knowledge maps, which makes Baidu unique in Chinese language processing.

The interviewed experts also pointed out that due to the particularity of Chinese, it is more difficult for Chinese enterprises to develop large-scale models, but if they break through, they will have greater advantages in providing local services.

Ding, a professor of artificial intelligence and business analysis at Lyon Business School in France, pointed out to the media a few days ago that language dialogue model training needs to make machines understand words, and English is slightly easier than Chinese. Ding explained that most of the Chinese characters processed by artificial intelligence technology in China are hieroglyphics, while English is explanatory and the characters are not particularly rich.

In addition, Lin, an assistant professor at the John Hopcroft Computer Science Center of Shanghai Jiaotong University, believes that in the future, the large language model will develop in a multi-modal and interactive direction, further integrating technologies in the fields of vision, pronunciation and reinforcement learning. Li Yanhong also said: "Multimodal transportation is an obvious development trend of generative artificial intelligence. In the future, with the enhancement of Baidu's multi-modal unified model, ERNIE Bot's multi-modal generation ability will continue to improve. "

In multimodal generation, Li Yanhong demonstrated ERNIE Bot's ability to generate text, pictures, audio and video. ERNIE Bot read a Sichuan dialect at the scene and made a video based on this text. However, Li Yanhong revealed that ERNIE Bot's video generation cost is high, and it is not open to all users at this stage, and it will be gradually accessed in the future.

Li Yanhong said that ERNIE Bot's training data includes: trillions of web pages, billions of search data and picture data, tens of billions of daily voice calls and 550 billion factual knowledge maps, which makes Baidu unique in Chinese language processing.

The interviewed experts also pointed out that due to the particularity of Chinese, it is more difficult for Chinese enterprises to develop large-scale models, but if they break through, they will have greater advantages in providing local services.

Ding, a professor of artificial intelligence and business analysis at Lyon Business School in France, pointed out to the media a few days ago that language dialogue model training needs to make machines understand words, and English is slightly easier than Chinese. Ding explained that most of the Chinese characters processed by artificial intelligence technology in China are hieroglyphics, while English is explanatory and the characters are not particularly rich.

In addition, Lin, an assistant professor at the John Hopcroft Computer Science Center of Shanghai Jiaotong University, believes that in the future, the large language model will develop in a multi-modal and interactive direction, further integrating technologies in the fields of vision, pronunciation and reinforcement learning. Li Yanhong also said: "Multimodal transportation is an obvious development trend of generative artificial intelligence. In the future, with the enhancement of Baidu's multi-modal unified model, ERNIE Bot's multi-modal generation ability will continue to improve. "

In multimodal generation, Li Yanhong demonstrated ERNIE Bot's ability to generate text, pictures, audio and video. ERNIE Bot read a Sichuan dialect at the scene and made a video based on this text. However, Li Yanhong revealed that ERNIE Bot's video generation cost is high, and it is not open to all users at this stage, and it will be gradually accessed in the future.

Before and after the conference, Baidu's share price experienced ups and downs. On March 16, the intraday share price of Hong Kong stock Baidu once expanded by more than 10% to 120. 1 HK$. At the close, Baidu's share price fell 6.36% to 125. 1 HK$. However, Baidu's share price has a strong momentum in the US stock market. On the same day, Baidu's US stocks opened lower and went higher, with an amplitude exceeding 7%. Closing Times 138. 16 USD, up 3.8%. On March 17, Baidu Hong Kong stocks performed strongly, with intraday gains exceeding 15%. As of the close of the day, Baidu Hong Kong stocks rose 13.67% to 142.2 Hong Kong dollars.

Within one hour after ERNIE Bot announced the opening of the invitation test, more than 30,000 enterprise users queued to apply for the API calling service test of ERNIE Bot Enterprise Edition, and the web pages that applied for product testing were crowded many times, and the traffic of official website and Baidu AI Cloud soared by a hundredfold.

ERNIE Bot's market fever continues to soar, and the capital market has also been revalued. Zhang Yi believes that this also represents the public's mood of "expectation, worry and hope" for the big language model/generative artificial intelligence.

No one can miss the scientific and technological revolution. In fact, "Are you really ready?" Not only for Baidu, but also a common public problem since this round of "ChatGPT" craze.

Li Yanhong observed that from 202 1, artificial intelligence technology began to change from "discrimination" to "generation".

Kai-Fu Lee, Chairman and CEO of innovation works, said at a trend sharing meeting on March 14 that the first phenomenal application in the AI 2.0 era was AIGC represented by GPT-4, also known as AI(Generative AI). Kai-fu Lee said that AI2.0 is a revolution that cannot be missed. This will be a huge platform opportunity, ten times larger than the mobile Internet. He also said that AI 2.0 is also China's first platform competition opportunity in the AI field.

Experts interviewed generally believe that AI companies all over the world have encountered a big problem before: even though the technical reserves are very rich, AI applications have not brought them rich benefits. The reason for this problem is that the application of AI products is mainly concentrated in B-end (enterprise users) and G-end (government users). When AI products enter enterprises or institutions, the process is often complicated, which will limit the rapid expansion of AI products in the market to some extent.

Therefore, Zhang Yi believes that the product application direction of AIGC is more likely to generate huge business opportunities at the C end. He analyzed that in the US market, before the C-end market was seized by companies such as Google, Amazon and Meta, Microsoft was under great pressure and needed a product to pull back. In the China market, Baidu has the same advantages as Google, such as powerful search engine's ability to grab data and the foundation of storage, sorting and analysis. China itself has a huge market of more than one billion people, and Baidu can do well.

"Baidu, Microsoft and Google are essentially competitions in two different markets, so I believe that ERNIE Bot and its series of products will definitely come out." Zhang Yi said.

Li Yanhong insists that ERNIE Bot is not a "tool for Sino-US scientific and technological confrontation". But he also admitted that the success of ChatGPT accelerated the progress of Baidu's launch of the product.

Wang Haifeng, CTO of Baidu, said that when human beings enter the AI era, the technology stack of IT technology can be divided into four layers: chip layer, frame layer, model layer and application layer. Baidu is one of the few artificial intelligence companies in the world with full stack layout at these four levels, and its self-developed technology leads the industry at all levels. For example, the high-end chip Kunlun Core, the deep learning framework of paddle flying, the large model of Wen Xin pre-training, and applications such as search, intelligent cloud, autonomous driving and small degree. Wang Haifeng believes that the advantage of Baidu's full-stack layout is that it can achieve end-to-end optimization in the four-tier architecture of technology stack, greatly improving efficiency.

Like ChatGPT, ERNIE Bot uses SFT (model fine tuning), RLHF (reinforcement learning from human feedback) and Prompt as the underlying technologies. In addition, ERNIE Bot has also adopted knowledge enhancement, retrieval enhancement and dialogue enhancement technologies. Wang Haifeng said that these three items are the re-innovation of Baidu's existing technological advantages.

Chen Duan believes that at a time when the integration of technological innovation is getting higher and higher, a single full-stack company has a comparative advantage in internal technology R&D co-ordination and later commercialization.

Self-confidence is very important, but the gap can not be ignored.

During the two sessions at the beginning of this month, Wang Zhigang, Minister of Science and Technology of China, responded to questions related to ChatGPT, taking football as an analogy, and pointed out that China still had a lot of work to do. "Playing football is dribbling and shooting, but it is not easy to be as good as Messi (soccer superstar Lionel Messi)."

Wang Zhigang pointed out that China has also done a lot of layout in this area, and the research in this area has been carried out for many years, and there are some.

As a result, "but it may remain to be seen to achieve the same effect as the current OpenAI," he added.

Wang Zhigang said that after ChatGPT came out, it attracted everyone's attention. In fact, from the source of technology itself, it is called NLP and NLU, which means natural language processing and natural language understanding. ChatGPT attracts people's attention because as a large model, it effectively combines big data, big computing power and strong algorithm, and its calculation method has been improved. The same principle is done differently. For example, everyone can make engines, but the quality is different.

However, whether it is ChatGPT or ERNIE Bot, the big language model behind it is core competitiveness. Zhao Dongyan, a researcher at Peking University Wang Xuan Computer Research Institute, told Caijing E Law that there is still a certain gap between domestic big models and OpenAI in terms of data, training methods and cost input.

A scientific and technological system person pointed out that objectively speaking, there is a big gap between China and the United States in basic research results in this field. These basic research achievements include natural language processing (NLP), database and GPU products. "If the United States cuts off the supply of GPU chips, the computing power (of China) will not keep up.".

The core of large-scale computing power lies in high-performance GPU chips. Zhou, an assistant professor at the School of Software, Beihang University, told Caijing E Law that the gap between China and the world in computing hardware such as GPU chips is about ten years, and the hardware level will seriously restrict the development of large-scale language models and scientific computing models.

Zhou believes that there is no generation gap between China's technology companies and OpenAI in technology and mode, and the gap is only within five years, and the gap in some smaller technical fields is only 2-3 years. In data collection, taking GPT-3 model as an example, Chinese only accounts for 5% of the training corpus. China science and technology enterprises have certain advantages in the accumulation of Chinese corpus, so it is very possible to achieve a breakthrough in the field of Chinese.

Giant 03' s next step: building ecology. How to make a profit on the big language model track represented by ChatGPT is a recognized problem for all parties.

OpenAI, which developed ChatGPT, is still a loss-making startup. From June, 5438 to October, 2023 10, an analysis report by Morgan Stanley, an investment bank, said that the response cost of ChatGPT was about 6 -28 times the average cost of Google search query.

However, Zhuang Du, a senior researcher at Tencent Research Institute and former vice president of Jingwei Venture Capital, believes that how much profit ChatGPT can bring is not the focus of OpenAI's attention, but what kind of services and applications can be developed based on its model, so as to build an ecosystem. "The development of ChatGPT needs an industrial ecology. For example, its integration with Microsoft-related applications is a good idea. " Cao Jianfeng said.

On March 5, local time, Yusef Medi, vice president and chief consumer marketing officer of Microsoft, issued a document saying that the new version of bing search engine has been running on GPT-4. OpenAI said that GPT-4 was trained on the Microsoft Azure AI supercomputer and will provide GPT-4 services to users around the world based on the Azure AI infrastructure.

Google announced the opening of the API interface of its big language model PaLM, and launched the developer-oriented tool MakerSuite. Through the PaLM API interface, developers can use PaLM to develop various applications. MakerSuite allows developers to quickly prototype their ideas. Over time, the tool will have the functions of rapid engineering, synthetic data generation and custom model adjustment.

Microsoft quickly followed suit. On March 16, local time, Microsoft announced that it would connect GPT-4 to Office family bucket. The new function is called "Microsoft 365 Copilot".

Li Yanhong said at the press conference that ERNIE Bot is positioned as an empowerment platform based on artificial intelligence, which will help the intelligent transformation of thousands of industries such as finance, energy, media and government affairs.

According to ERNIE Bot's invitation test scheme, the first batch of users can experience the products in official website, ERNIE Bot from March 16, and will be opened to more users one after another. In addition, Baidu AI Cloud will soon open ERNIE Bot API interface calling service to corporate customers. This service will be accepted by appointment from March 16.

As of the morning of March 1 1, the number of enterprise users queuing to apply for Baidu AI Cloud ERNIE Bot Enterprise Edition API call server test has increased to 90,000, and Baidu has received 6,588 inquiries about cooperation in ERNIE Bot.

Chen Duan believes that this round of competition is not only the competition of commercial subjects, but also the next round of national digital competitiveness. Therefore, Baidu's top priority is not entirely technology research and development, but also needs to lead more start-ups and ecological partners to join the ecological camp.

In Chen Duan's view, China has advantages in establishing an ecosystem. Chen Duan pointed out that after years of development, the supporting innovation of application layer ecology in cmnet is very mature. Many small and medium-sized entrepreneurial teams in the application layer have done a lot of local and vertical scene-side innovations in cooperation with the mobile Internet ecosystem in the past, and it is still applicable to migrate this model and underlying infrastructure from the mobile Internet to the big model field.

Are there still opportunities for SMEs? Facing the wave of big language model, how should China enterprises seize opportunities and avoid risks?

In China, there are two types of enterprises deploying ChatGPT: the first is the traditional big Internet companies, and the second is some start-ups.

Chen Duan believes that start-ups in the market have missed the initial stage of building a big model. Chen Duan analyzed that,

Rebuilding a generative artificial intelligence enterprise is closely related to the opportunity, the underlying ecological support, the founder's own experience, experience, vision and the natural mobilization ability of personal IP. In addition, the input of the early large model, whether it is computing power or other costs, and the time window are very important.

Chen Duan said that at present, Baidu has the ability to coordinate its other products with ERNIE Bot, just like Microsoft launched Copilot with Office and GPT-4, but "if there is no ecological support, it is very problematic for entrepreneurs to simply make big models".

Zhang Yi also believes that for enterprises with financial and strength support, building large-scale products alone may be more favored by capital and entrepreneurs. But for small and medium-sized enterprises, it is also a good choice to rely on ERNIE Bot's open platform to graft their own applications in the segmentation field.

Because it takes a long time and huge investment to make a big language model.

Behind the success of OpenAI is Microsoft's huge investment over the years. On June 23rd, 20231October 23rd, US time, Microsoft announced that it would invest billions of dollars in OpenAI for several years. 20 19 and 202 1, Microsoft invested in OpenAI twice. The investment in 20 19 was $65,438 billion, while the investment in 20021year was not disclosed.

Yuan Xingyuan, founder of Cai Yun Science and Technology of AI Company, pointed out in an interview with 36Kr that if you want to run a model with more than 65.438+000 billion parameters at a time, you should at least reach the level of "kilocalories per month", that is, you should use 654.38+0000 GPU cards and then train for one month. Even if you don't use the most advanced NVIDIA A 100, according to the average price of 50,000 yuan for a GPU, 1000 GPU means the calculation cost of 50 million yuan per month, which is not counting algorithm engineer's salary.

"No matter which company, it is impossible to make such a big language model in a few months." Li Yanhong said at the press conference that deep learning and natural language processing need years of persistence and accumulation and cannot be accelerated. Large-scale model training can be called violent aesthetics, which requires a lot of computing power, big data and big models, and the cost of each training task is very high.

According to the data provided by Baidu, Baidu's accumulated R&D investment in the past decade has exceeded 654.38+000 billion yuan. In 2022, Baidu's core R&D expenditure was 2 1.4 1.6 billion yuan, accounting for 22.4% of Baidu's core revenue. However, Baidu did not disclose the proportion of large model research and development in core research and development expenses.

Li Yanhong said at the press conference that Baidu's positioning of ERNIE Bot is a universal empowerment platform, and thousands of industries such as finance, energy, media and government affairs can realize intelligent changes, improve efficiency and create huge commercial value based on this platform. Li Yanhong believes that there will be three major industrial opportunities in the big model era, namely, new cloud computing companies, companies that fine-tune industry models, and companies that develop applications based on big model libraries, namely, application service providers.

Li Yanhong asserted that for most entrepreneurs and enterprises, the real opportunity is not to build basic big models like ChatGPT and ERNIE Bot from scratch, which is unrealistic and uneconomical. It may be the real opportunity to develop important application services based on the general language model first. At present, based on text generation, image generation, audio generation, video generation, digital people, 3D and other scenes, many entrepreneurial star companies have emerged, which may be the new giants in the future.

"The final product form of big model and generative artificial intelligence is still unknown, so this road is destined to be a long-distance running, which requires the close and continuous follow-up of the entire scientific and technological community in capital, R&D and model innovation." Zhang Yi said.

Kai-fu Lee believes that AI2.0 will be first applied in the field of fault tolerance, and there is no doubt that the biggest application field now is content creation. Every field can rewrite the original App once to create a more profitable business model. In the end, the generation ability of AI2.0 reduces the cost to almost zero.