What is Baidu Wenxinyiyan? What do you expect from Wen Xinyiyan?

Baidu Wenxin Yiyan is an ancient poem updated daily, aiming to stimulate readers' thinking and enhance the literary soul of literati. I hope that Wen Xin Yi Yan can bring me more wonderful literary experiences, deeper cultural artistic conceptions, and more interesting historical stories.

In mid-March, global technology giants are once again competing to debut on the large language model track.

Within a week, OpenAI, an American startup that developed ChatGPT, Microsoft, the technology giant that has invested heavily in OpenAI, and Baidu, the leading Chinese Internet company, have successively released the latest in the field of large language models (LLM). dynamic. This has once again triggered global attention in this field.

On March 14, local time, OpenAI announced the latest version of its large-scale language model-GPT-4, which has significantly improved question and answer quality and technology compared to GPT-3.5.

On the afternoon of March 16, Baidu launched the test of Wenxinyiyan, a new generation large language model and generative AI product, becoming the first Chinese company to join the competition on this track.

At the press conference, Robin Li, founder, chairman and CEO of Baidu, demonstrated Wen Xinyiyan’s role in literary creation, business copywriting creation, mathematical calculations, Chinese understanding, and multi-modality through a question and answer session. Five usage scenarios including dynamic generation. A few hours later, Microsoft announced that it would integrate GPT-4 into the Office family bucket, with a new name of "Microsoft 365 Copilot".

As mentioned in the article published by Caijing E Law on February 17 (OpenAI exclusive response | Why is ChatGPT not open to all Chinese users for registration?), mobile phone numbers in mainland China and Hong Kong cannot register for ChatGPT account. In addition, although OpenAI’s application programming interface (API) is open to 161 countries and regions, it does not include mainland China and Hong Kong.

On the one hand, the industry is generally concerned about who will be the next trend-setter in the unstoppable technological wave of AIGC (generative artificial intelligence)? On the other hand, during this sensitive period of technological competition between China and the United States, all parties are also paying close attention to the ripples caused by Baidu's first step and how Chinese companies should respond. 01 "Are you really ready?"

On March 16, Robin Li gave a speech wearing a white shirt and sneakers. He faced the question directly at the beginning, "Recently, many friends have asked me, why is it today? Are you really ready?"

Robin Li’s answer is that although Baidu has invested in AI research for more than ten years and has made full preparations for the release of Wen Xin Yi Yan, it “cannot be said to be completely ready” because Wen Xin Yi Yan benchmarks against ChatGPT , and even GPT-4, the threshold is very high and there are "many imperfections." But he emphasized that "once there is real human feedback, Wen Xinyiyan's progress will be very fast."

Robin Li explained that the reason why he chose to release it on the same day was because there was market demand: customers and partners wanted to use the latest and most advanced large language models earlier.

How to understand what Robin Li said: "The threshold for benchmarking GPT-4 is very high"?

On March 14, local time, OpenAI announced the latest version of its large-scale language model-GPT-4. It is worth noting that GPT-4 is a large multi-modal model, that is, it can accept image and text type input. GPT-3.5, on the other hand, can only accept text input.

In the demo video, OpenAI president and co-founder Greg Brockman draws a sketch of a website with pen and paper and imports the image into GPT-4. After only 1 to 2 seconds, GPT-4 generated the web page code and produced a website that was highly similar to the sketch. According to experimental data released by OpenAI, the GPT-4 model has made great progress compared with the previous generation GPT-3.5, and has performed better than the vast majority of humans in many professional tests.

Pan Helin, co-director of the Digital Economy and Financial Innovation Research Center of the International Joint Business School of Zhejiang University, believes that Wen Xinyiyan still needs to be fully opened in the future to gain user testing. Whether it is open to C-side users through B-side APIs or directly, user experience and reputation are the last word. Currently, ChatGPT is not open to Chinese users. In the domestic market, Baidu will gain first-mover advantage.

Zhang Yi, CEO and chief analyst of iiMedia Consulting, who has evaluated both OpenAI and Baidu products, said that the GPT series of large models, including GPT-4 and Wenxin Yiyan, are essentially the same type Products only differ in their respective data coverage and data model accumulation. In the short term, OpenAI's product preparation time is relatively sufficient, and its intelligence is temporarily ahead. But for Wen Xinyiyan, it is also very remarkable to be able to train such a product in such a short period of time.

At the same time, Zhang Yi is also more confident that Baidu will make better products. His reason is that China will have more advantages in terms of talent reserves for artificial intelligence, big data, and large models.

Chen Duanze, director of the Digital Economy Integration Innovation and Development Center of the Central University of Finance and Economics, believes that compared with overseas competitors, Baidu’s biggest advantage is that it is based on the local market and has built a moat of language and cultural understanding.

As a large language model product developed by a Chinese company, Wen Xinyiyan’s Chinese understanding ability has attracted much attention. The important reason is that many commentators previously believed that ChatGPT’s Chinese question and answer capabilities are not as strong as its English question and answer capabilities.

Robin Li said that as a large language model rooted in the Chinese market, Wenxinyiyan has the most advanced natural language processing capabilities in the Chinese field. During the on-site demonstration, Wen Xinyiyan correctly explained the meaning of the idiom "Luoyang paper is expensive" and the economic theory corresponding to "Luoyang paper is expensive", and also created an acrostic poem using "Luoyang paper is expensive".

Robin Li said that Wen Xinyiyan’s training data includes: trillions of web page data, billions of search data and image data, tens of billions of daily voice call data, and 550 billion facts. Knowledge graph, etc., which makes Baidu unique in processing Chinese language.

Interviewed experts also pointed out that due to the particularity of the Chinese language, Chinese companies face greater difficulties in developing large models. However, if they can make a breakthrough, they will have greater advantages in providing local services. .

Ding Wenxuan, a professor of artificial intelligence and business analysis at Lyon Business School in France, recently pointed out to the media that language dialogue model training requires machines to understand text, and English is slightly easier than Chinese. Ding Wenxuan explained that most of the Chinese language processed by China's artificial intelligence technology is pictographic, while English is explanatory and is not particularly rich in words in comparison.

In addition, Lin Zhouhan, assistant professor at the John Hopcroft Computer Science Center of Shanghai Jiao Tong University, believes that in the future, large language models will most likely develop in a multi-modal and interactive direction, further Technologies in vision, speech, reinforcement learning and other fields are integrated. Robin Li also said: "Multimodality is a clear development trend of generative AI. In the future, as Baidu's ability to unify large multimodal models increases, Wen Xinyiyan's multimodal generation capabilities will continue to improve."< /p>

In terms of multi-modal generation, Robin Li demonstrated Wen Xin Yi Yan’s ability to generate text, pictures, audio and video. Wen Xinyiyan read a piece of content in Sichuan dialect at the scene and generated a video based on the text. However, Robin Li revealed that the production cost of Wen Xinyiyan's videos is relatively high, and it is not open to all users at this stage, but will be gradually accessed in the future.

Robin Li said that Wen Xinyiyan’s training data includes: trillions of web page data, billions of search data and image data, tens of billions of daily voice call data, and 550 billion facts. Knowledge graph, etc., which makes Baidu unique in processing Chinese language.

Interviewed experts also pointed out that due to the particularity of the Chinese language, Chinese companies face greater difficulties in developing large models. However, if they can break through, they will have greater advantages in providing local services. .

Ding Wenxuan, a professor of artificial intelligence and business analysis at Lyon Business School in France, recently pointed out to the media that language dialogue model training requires machines to understand text, and English is slightly easier than Chinese. Ding Wenxuan explained that most of the Chinese language processed by China's artificial intelligence technology is pictographic, while English is explanatory and is not particularly rich in words in comparison.

In addition, Lin Zhouhan, assistant professor at the John Hopcroft Computer Science Center of Shanghai Jiao Tong University, believes that in the future, large language models will most likely develop in a multi-modal and interactive direction, further Technologies in vision, speech, reinforcement learning and other fields are integrated. Robin Li also said: “Multimodality is a clear development trend of generative AI.

In the future, as Baidu's multi-modal unified large model capabilities increase, Wen Xinyiyan's multi-modal generation capabilities will continue to improve. ”

In terms of multi-modal generation, Robin Li demonstrated Wen Xinyiyan’s ability to generate text, pictures, audio and video. Wenxinyiyan read a piece of content in Sichuan dialect at the scene, and based on the text A video was generated. However, Robin Li revealed that Wen Xinyiyan’s video production cost is high and it is not open to all users at this stage. It will be gradually accessed in the future.

Baidu’s stock price experience before and after the press conference. On March 16, Baidu's stock price fell by more than 10% during the session, to HK$120.1. As of the close, Baidu's stock price fell by 6.36%, to HK$125.1. However, Baidu's stock price had strong momentum on the US stock market that day. Baidu's U.S. stocks opened lower and moved higher, with an amplitude of over 7%. As of the close, Baidu's Hong Kong stocks performed strongly, rising by more than 15% as of the close of the day. The increase was 13.67%, reported at 142.2 Hong Kong dollars.

Within one hour after Wen Xin Yiyan announced the launch of the invitation test, more than 30,000 corporate users had queued up to apply for the Wen Xin Yiyan Enterprise Edition API call service test. The application page for product testing has been crowded many times, and the traffic on Baidu Smart Cloud's official website has surged a hundred times.

Wen Xinyiyan's market popularity continues to surge, and the capital market has also revalued its value. Zhang Yi believes that this is also the case. It represents the public’s “expectation, worry, and hope” towards big language models/generative AI. 02 A technological revolution that no one can miss.

In fact, “Is it really ready?” " is not just for Baidu, but also a common question among the public since this round of "ChatGPT" craze.

Robin Li observed that starting from 2021, artificial intelligence technology will begin to shift from "discriminative" to "generative" "Change.

Kaifu Lee, Chairman and CEO of Innovation Works, said at a trend sharing meeting on March 14 that the first phenomenal application in the AI ??2.0 era is AIGC represented by GPT-4 , also known as Generative AI. Kaifu Li said that AI2.0 is a revolution that cannot be missed. It will be a huge platform opportunity, which will be ten times larger than the mobile Internet. He also said, AI 2.0 is also China’s first platform competition opportunity in the field of AI.

Experts interviewed generally believe that AI companies around the world have encountered a huge problem: even with abundant technical reserves, AI The application has not brought them huge benefits. The reason for this problem is that the application of AI products is mainly concentrated on the B-side (enterprise users) and G-side (government users), and AI products often go through a process when entering enterprises or institutions. Complex, this will to some extent limit the rapid expansion of AI products in the market. Therefore, Zhang Yi believes that AIGC’s product application direction is more likely to generate huge business opportunities on the C side. Analysts say that in the U.S. market, the C-end market has previously been occupied by companies such as Google, Amazon, and Meta, which has put great pressure on Microsoft, and it needs a product to make a comeback. In the Chinese market, Baidu has the same advantages as Google. It has a strong search engine ability to capture data, as well as storage, organization, and analysis capabilities. China itself has a huge market of more than one billion people, and Baidu can do very well.

"Baidu. Competition with Microsoft and Google is essentially two different markets, so I believe Wen Xinyiyan and its series of products will definitely come out. "Zhang Yi said.

Robin Li insisted that Wen Xinyiyan is not "a tool for Sino-US technological confrontation." But he also admitted that the success of ChatGPT has accelerated Baidu's launch of the product.

Baidu CTO Wang Haifeng said that when humans enter the AI ??era, the technology stack of IT technology can be divided into four layers: chip layer, framework layer, model layer and application layer. Baidu is one of the few companies in the world that operates on these four layers. A full-stack artificial intelligence company with industry-leading self-developed technologies at all levels, such as high-end chip Kunlun core, Feipiao deep learning framework, Wenxin pre-trained large model, as well as search, intelligent cloud, autonomous driving, and Xiaodu. and other applications. Wang Haifeng believes that the advantage of Baidu's full-stack layout is that it can achieve end-to-end optimization in the four-layer architecture of the technology stack and greatly improve efficiency.

Wenxinyiyan, like ChatGPT, uses SFT (model fine-tuning), RLHF (reinforcement learning from human feedback) and Prompt (prompts) as underlying technologies. In addition, Wen Xinyiyan also uses knowledge enhancement, retrieval enhancement and dialogue enhancement technologies. Wang Haifeng said that these three items are re-innovations of Baidu’s existing technological advantages.

Chen Duan believes that at a time when technological innovation is becoming more and more integrated, a single company with a full-stack layout has comparative capabilities in terms of internal technology R&D coordination capabilities and later-stage commercialization collaborative capabilities. Advantages.

Confidence is important, but the gap cannot be ignored.

During the two sessions earlier this month, China’s Minister of Science and Technology Wang Zhigang used a football analogy in response to questions related to ChatGPT, pointing out that China still has a lot of work to do. "Playing football is all about dribbling and shooting, but it is not easy to be as good as Messi (football superstar Lionel Messi)."

Wang Zhigang pointed out that China is also in this regard A lot of layouts have been made, research in this field has been carried out for many years, and there have been some results. "But it may still have to wait and see to achieve results like OpenAI," he added.

Wang Zhigang said that after ChatGPT came out, it attracted everyone's attention. In fact, from the source of the technology itself, it is called NLP and NLU, which are natural language processing and natural language understanding. The reason why ChatGPT has attracted attention is that as a large model, it effectively combines big data, large computing power, and powerful algorithms, and its calculation methods have made progress. The same principle, done differently. For example, everyone can make an engine, but the quality is different.

However, whether it is ChatGPT or Wenxinyiyan, the large language model behind it is the core competitiveness. Zhao Dongyan, a researcher at the Wangxuan Institute of Computer Science at Peking University, told Finance E Law that there is still a certain gap between domestic large models and OpenAI in terms of data, training methods and cost investment.

A person in the science and technology system pointed out to Finance E Law that objectively speaking, there is a large gap between China and the United States in basic research results in this field. These basic research results include natural language processing (NLP), databases, and GPU products. "The United States cuts off the (supply of) GPU chips, and (China's) computing power cannot keep up."

The core of large-scale computing power lies in high-performance GPU chips. Zhou Haoyi, an assistant professor at the School of Software at Beihang University, told Finance E Law that in terms of computing hardware such as GPU chips, the gap between China and the world is about ten years. The level of hardware will seriously restrict the development of large language models and scientific computing models.

Zhou Haoyi believes that in terms of technology and models, there is no generation gap between Chinese technology companies and OpenAI. The gap is only within five years, and the gap in some smaller technical fields is only 2-3 years. In terms of data collection, taking the GPT-3 large model as an example, Chinese accounts for only 5% of its training corpus. Chinese technology companies have certain advantages in accumulating Chinese corpus, so they are very likely to achieve breakthroughs in the Chinese field. 03 Giant’s next step: building an ecosystem

How to achieve profitability in the large language model track represented by ChatGPT is a problem recognized by all parties (Cold thinking on the explosion of ChatGPT: profitability problems and governance challenges).

OpenAI, which developed ChatGPT, is still a loss-making startup. In January 2023, an analysis report by investment bank Morgan Stanley stated that the cost of a reply to ChatGPT is approximately 6 to 28 times the average cost of a Google search query.

However, Cao Jianfeng, a senior researcher at Tencent Research Institute, and Zhuang Minghao, former vice president of Matrix Partners, both believe that how much profit ChatGPT can bring is not the focus of OpenAI, but what can be grown based on its model. services and applications to build an ecosystem. "The development of ChatGPT requires an industrial ecosystem. For example, its integration with Microsoft-related applications is a very good idea." Cao Jianfeng said.

On March 15, local time, Microsoft Vice President and Chief Consumer Marketing Officer Joseph Medi issued a message stating that the new version of the Bing search engine is already running on GPT-4.

According to OpenAI's disclosure, GPT-4 is trained on Microsoft's Azure AI supercomputer, and GPT-4 services will be provided to users around the world based on Azure's AI infrastructure.

Google announced the opening of the API interface of its large language model PaLM and launched MakerSuite, a tool for developers. Through the PaLM API interface, developers can use PaLM for the development of various applications. MakerSuite allows developers to quickly prototype their ideas, and over time, the tool will have capabilities for rapid engineering, synthetic data generation, and custom model tuning.

Microsoft quickly followed suit. On March 16, local time, Microsoft announced that it would integrate GPT-4 into Office Family Bucket. The new feature is called "Microsoft 365 Copilot."

Robin Li said at the press conference that Wenxinyiyan is positioned as an artificial intelligence base-type empowerment platform that will help intelligent transformation of various industries such as finance, energy, media, and government affairs.

According to Wen Xin Yiyan’s invitation test plan, starting from March 16, the first batch of users can use the invitation test code to experience the product on Wen Xin Yiyan’s official website, and it will be opened to more users in the future. In addition, Baidu Smart Cloud will soon open Wenxinyiyan API interface calling services to enterprise customers. The service will be open for reservations from March 16.

As of 11 a.m. on March 18, the number of corporate users queuing up to apply for the API call server test of Baidu Smart Cloud Wenxin Yiyan Enterprise Edition has increased to 90,000. Baidu has received notifications about Wenxin Yiyan’s cooperation. Consult 6588 items.

Chen Duan believes that this round of competition is not only a competition between commercial entities, but is actually about the next round of national digital competitiveness. Therefore, Baidu's top priority is not only technical research and development, but also the need to lead more start-up companies and ecological partners to join the ecological camp.

In Chen Duan’s view, China has advantages in building an ecosystem. Chen Duan pointed out that after years of development of China's mobile Internet, the supporting innovation of application layer ecology has become very mature. Many small, medium and micro entrepreneurial teams at the application layer have made a lot of local and vertical scene-side innovations in conjunction with the mobile Internet ecosystem in the past. It is still applicable to migrate the past model and underlying infrastructure from mobile Internet to the large model field. 04Are there still opportunities for small and medium-sized enterprises?

Faced with the wave of big language models, how can Chinese companies seize opportunities and avoid risks?

In China, there are two types of companies deploying ChatGPT: the first is traditional large Internet companies, and the second is some start-ups.

Chen Duan believes that the current start-up companies on the market have missed the initial entrepreneurial stage of laying out large models. Chen Duan analyzed that re-building a generative AI company is closely related to timing, the underlying ecological support, as well as the founder’s own experience, experience, vision, and the natural ability to mobilize personal IP. . In addition, the initial investment in large models, whether it is computing power or other costs, and the time window are very important.

Chen Duan said that currently, Baidu has the ability to synergize its other products with Wenxinyiyan, just like Microsoft collaborated with Office and GPT-4 to launch Copilot, while "entrepreneurs simply go for big business." The model does not have a supporting ecosystem, which is very problematic."

Zhang Yi also believes that for companies with financial and strength support, building large-scale model products alone may be more favored by capital and entrepreneurs. But for small and medium-sized enterprises, it is also a good choice to rely on Wen Xinyiyan's open platform to graft their own applications in subdivided fields.

Because it takes a long time and a huge amount of money to make a large language model.

Behind the success of OpenAI is Microsoft’s huge investment over the years. On January 23, 2023, U.S. time, Microsoft announced a multi-year investment worth billions of dollars in OpenAI. In 2019 and 2021, Microsoft invested in OpenAI twice. Investment in 2019 was $1 billion, while investment in 2021 was an undisclosed amount.

Yuan Xingyuan, the founder of the AI ??company "Caiyun Technology" pointed out in an interview with 36 Krypton that if you want to run through a model with more than 10 billion parameters, you must at least achieve the level of "kilocalories/month" , that is: use 1000 GPU cards, and then train for a month. Even if the most advanced Nvidia A100 is not used, based on the average price of a GPU of 50,000 yuan, 1,000 GPUs means a computing power cost of 50 million yuan per month, not counting the salary of algorithm engineers.

“No matter which company it is, it is impossible to build such a large language model by just a few months.” Robin Li said at the press conference that deep learning and natural language processing require many years of research. Persistence and accumulation cannot be achieved quickly. Large model training can be called violent aesthetics. It requires large computing power, big data and large models. Each training task is expensive.

Data provided by Baidu shows that Baidu has invested more than 100 billion yuan in R&D in the past ten years. Baidu's core R&D expenses in 2022 will be 21.416 billion yuan, accounting for 22.4% of Baidu's core revenue. However, Baidu did not disclose the proportion of large model R&D in core R&D expenses.

Robin Li said at the press conference that Baidu’s positioning of Wen Xinyiyan is a universal empowerment platform. Thousands of industries such as finance, energy, media, and government affairs can all be implemented based on this platform. Intelligent transformation improves efficiency and creates huge business value. Robin Li believes that the era of large models will generate three major industry opportunities, namely new cloud computing companies, companies that fine-tune industry models, and companies that develop applications based on large model bases, that is, application service providers.

Robin Li asserted that for most entrepreneurs and companies, the real opportunity is not to build basic large-scale models like ChatGPT and Wenxinyiyan from scratch. This is very unrealistic and uneconomical. This may be the real opportunity to preemptively develop important application services based on a general large language model. At present, based on text generation, image generation, audio generation, video generation, digital people, 3D and other scenarios, many entrepreneurial star companies have emerged, which may be new giants in the future.

“The final product form of large models and generative AI is still unknown, so this road is destined to be a long-distance race, requiring the entire technology community to closely and continuously follow in terms of capital, R&D, and model innovation. "Zhang Yi said.

Li Kaifu believes that AI2.0 will be first applied in areas that can tolerate errors, and there is no doubt that the largest application area is content creation. In each field, the original App can be rewritten to create a more profitable business model. Ultimately, the generation capabilities of AI 2.0 will reduce costs to almost zero.