Today (April 19), American tech giant Meta launched Llama 3, which is billed as "the most powerful open-source model ever", which can be used directly by external developers for free.
In the eyes of the outside world, Meta is now launching Llama 3 to catch up with industry leader OpenAI.
However, Meta CEO Mark Zuckerberg told foreign media, "Our goal is not to compete with open-source models, but to surpass everyone and build the most advanced artificial intelligence." ”
01
The best performing open-source model
It's for GPT-4
Obviously, Meta's launch of Llama3 this time is aimed at GPT-4.
As part of its catch-up efforts, Meta has been releasing models like Llama 3 for developers to use commercially for free, as the success of the powerful free-to-play model could hinder competitors' plans to earn revenue from its proprietary technology.
Zuckerberg said that Llama 3 is a huge improvement over Llama 2 due to pre-training and instruction fine-tuning.
Most of the main highlights focus on: model architecture, pre-training data, pre-training scale, and instruction fine-tuning.
For example, compared with Llama 2, the training set size of Llama3 is 7 times larger, the amount of code data is increased by 4 times, and the training efficiency is improved by about 3 times.
To put it simply, Llama3 currently comes in two versions: version 8B and version 70B.
According to Meta's official statement, these two versions are currently the best performing open source models of the same volume. Major cloud providers will also be available in the near future.
Among them, the 70B version has more than 400 billion parameters and will be directly benchmarked against GPT-4, while the 8B version has better performance than the previous version of Llama 2 70B in some test sets.
In addition, the 8B version outperformed the Gemma-7B and Mistral-7B versions, while the Llama 3 70B version also outperformed the Gemini 1.5 Pro and Claude 3 Sonnet in many ways.
Judging from the feedback from the bigwigs in the AI field, Llama 3's performance this time is indeed unusual.
Yann LeCun, one of the AI triumvirate, made a post specifically for Llama 3, and Musk appeared in the comment section with a "Not bad" message.
Andrew Ng, one of the most authoritative scholars in the field of artificial intelligence and machine learning and an advocate of AI open source, said: "The release of Llama 3 is the best gift I have ever received in my life, thank you Meta!"
02
Official version of Llama 3
It may be released in July
Meta is likely to launch more versions in the coming months.
Jim Fan, a senior scientist at NVIDIA, believes that the release of Llama 3-400B and above in the future may be a "watershed" of sorts, and the open source community may soon be able to use GPT-4-level models.
Meta also made it clear on its official blog that the official version of Llama3, which will be launched in the next few months, will have "multimodal" features, that is, it can control both text and image generation.
However, according to people familiar with the matter, the researchers have not yet done detailed fine-tuning work on Llama3, so it has not yet been decided whether Llama 3 will be a multimodal model.
Fine-tuning is a crucial step in the model development process, by injecting additional data into an existing model so that it can acquire new knowledge or adapt to specific task needs. Typically, models with larger numbers of parameters produce higher quality output, while smaller models are known for their fast responses.
To go into more detail, Meta also plans to roll out new features, longer context windows, additional model sizes, and enhanced performance, and will share research papers on Llama 3.
It is reported that the official version of Llama 3 will be launched in July this year.
03
Zuckerberg is bound to Google
Compete with OpenAI and Microsoft
Zuckerberg also told investors earlier this month that the main areas of focus this year include the launch of Llama 3 and "expanding the usefulness of Meta's AI assistant."
So, in addition to the release of Llama 3, Meta also announced a new strategic partnership with Alphabet's Google.
The partnership will enable Meta's AI assistant to incorporate authoritative results from Google search in real-time when answering user questions, which is also an effective addition to the existing partnership with Microsoft Bing. Subsequently, Meta AI assistant is expanding to more than a dozen markets outside of the United States, including Australia, Canada, Singapore, Nigeria, and Pakistan.
Zuckerberg said at a press conference on Thursday that Meta AI is "the smartest AI assistant you can freely use." He said the largest version of Llama 3 is currently being trained with 400 billion parameters and scores 85 points on the MMLU (Massive Multitasking Language Understanding) test. He said the two smaller versions now have 8 billion and 70 billion parameters, respectively, with the latter scoring 82 points for MMLU.
Currently, Meta has integrated the Llama 3 model into Meta AI, which is officially considered to be the world's leading AI assistant. The web version is now live: meta.ai, users can save conversations with Meta AI when they log in.
According to reports, users can use Meta AI on Facebook, Instagram, WhatsApp, and Messenger to complete tasks, learn, and create.
04
The controversy between the open and closed sources has intensified again
Zhou Hongyi replied to Robin Li
The debate between open source and closed source has gradually turned into a religious battle of beliefs, and it is difficult for anyone to remain neutral.
Not long ago, at the Baidu AI Developer Conference, Robin Li, the founder, chairman and CEO of Baidu, said, "In the past, it was said that open source was cheap, but in fact, in the field of large models, open source is the most expensive." So the open source model will fall further and further behind. ”
Previously, Robin Li also mentioned in his internal speech that the open source model is difficult to achieve a high level of firewood for everyone; under the same ability, closed source has more advantages in cost; closed source will continue to lead, rather than lead for a while; closed source can have a real business model, and only by making money can it gather talents and computing power.
In this regard, Zhou Hongyi, the founder of 360 Group, was suspected of replying to Robin Li at the 27th Harvard China Forum:
"I have always believed in the power of open source, and as for the nonsense of some celebrities on the Internet, don't be fooled by them, saying that open source is better than closed source. ”
In a word, today there is no Linux without open source, and there is no Internet without Linux, and even the company itself has grown to this day with the help of open source. ”
"The number of engineers and scientists gathered in the source community is hundreds of times higher than that of the closed source. Therefore, this year's open-source has exceeded the capabilities of GPT-3.5 after only one year. In the next year or two, the power of open source is likely to reach or exceed the level of closed source. ”
Zhou Hongyi also gave an example, "Two days ago, Baidu's Robin Li said that their large model surpassed GPT-4, and then Wang Xiaochuan didn't believe it, so he came out to scold Robin Li." In fact, it makes sense for you to listen carefully to what Robin Li said, he is saying that in terms of writing ancient poems, Baidu surpasses GPT-4."
Recently, Wang Xiaochuan, the former founder of Sogou and the current founder of Baichuan Intelligence, sharply complained in an interview with the media: Robin Li is very magical, and he shouted in February last year that he was only two months behind OpenAI, which is enough to hallucinate.
Abroad, Yann LeCun, one of the three giants of AI, believes that the free exchange of scientific papers and code, as well as the open sharing of AI training models, have enabled the United States to maintain its leading position in the field of science and technology. This concept is not new, it has been around for a long time.
Open source democratizes access. It gives more people and businesses the ability to take advantage of state-of-the-art technology and compensate for potential weaknesses. It also helps to promote democratic values and institutions, reduce social disparities and improve competition.
Scientists from the Massachusetts Institute of Technology and the University of Cambridge mentioned in a paper that they found that open source models do have the risk of being manipulated by bad actors. Researchers at Anthropic have also published a paper warning that AI poisoning could lead to open-source models turning into latent spies.
05
Today, some media wrote:
Compared with the dispute between open and closed sources at home and abroad, more importantly, in the process of changing again and again, we stand on the threshold of technological change and can get a glimpse of AI technology is no longer a cold algorithm and data pile, but has multiple perception capabilities and accurate social understanding. It heralds a future where artificial intelligence will be more deeply integrated into our lives.
This kind of integration may seem a little bleak in the debate between open source and closed source. But at this moment, hearing different voices and positions, and feeling the fierce collision brought about by technological progress, may be the meaning of technology itself.
Finally, what are your thoughts on the debate between open source and closed source of large models?
[The author of this article is a dark horse, an entrepreneur who originally wrote.] If you need to reprint, please contact the WeChat public account (ID: iheima) for authorization. ］