Author | Yun Peng
Edit | Li Shuiqing
Zhidong news on May 11, early this morning, Apple AI fried the pot!
On the one hand, Bloomberg revealed the news: Apple is about to sign an agreement with OpenAI to use ChatGPT-related technology in the iPhone.
On the other hand, the "New York Times" revealed a series of blockbuster generative AI upgrades that Apple's Siri is about to release, the details of the research and development behind it, and the reasons behind the decision of Apple's top executives to do a "brain transplant" for Siri.
Foreign media AppleInsider reported that Apple will use iconic generative AI functions such as real-time recording transcription and text summarization in core system-level applications such as voice memos, notes, messages, and Safari to do "system-level AI".
In the current wave of generative AI boom based on large language models, Apple is undoubtedly lagging behind, and chatbot applications represented by ChatGPT have become popular all over the world, but Apple has not yet made a clear move.
Executives, such as Apple CEO Tim Cook, kept conveying to the outside world that "Apple takes generative AI seriously", but in the end, everyone focused on how Apple would do AI.
Today, more key questions about Apple's AI seem to have clearer answers. We can already get a glimpse of Apple's generative AI move.
1. Embrace OpenAI? Apple does AI choose to walk on two legs
Is Apple doing AI alone or cooperatively? At present, Apple is likely to "walk on two legs".
According to Bloomberg, Apple is about to reach a deal with OpenAI to use OpenAI's related technology on the iPhone to put ChatGPT on the iPhone. Under the agreement, Apple will use ChatGPT features in the future release of iOS 18.
Negotiations between the two sides are still ongoing, and an agreement has not yet been finalized.
However, it is worth noting that ChatGPT has been open for download in Apple's App Store for a long time, and the use of ChatGPT on the iPhone is already in the past, so the cooperation between Apple and OpenAI may focus more on the model level behind it, and integrate more "ChatGPT's capabilities" into the iPhone.
According to Bloomberg, Apple is also in talks with Google about using the Gemini model.
Second, Apple wants to "change the brain" for Siri! Heal from the roots
With the cooperation out of the way, let's take a look at what Apple's own AI is going to do. According to the New York Times, Siri will be one of Apple's key tools for generative AI.
According to reports, in fact, since the beginning of last year, that is, when ChatGPT was just popular, Craig Federighi, Apple's senior vice president of software engineering, and John Giannandrea, Apple's senior vice president of machine learning and AI strategy, have made a decision to give Siri a "brain transplant".
That's right, it's no longer possible to treat the symptoms, and Apple wants to directly change Siri's brain.
These two executives are familiar faces at Apple's press conference, and they are also Apple's core executives who report directly to Cook, so it can be said that Cook and these two people are the core decision-makers of what Apple's AI will become.
The two executives reportedly spent weeks testing OpenAI's ChatGPT, prompting them to make the decision to give Siri a "brain swap."
An Apple source familiar with Siri revealed that ChatGPT was able to write poetry, code, and answer complex questions almost anything at the time, which made Siri look a little outdated.
3. Thirteen years after Siri, Apple launched the largest internal reshuffle in a decade
In 2011, Siri appeared on the iPhone as one of the first intelligent assistants, but at that time "intelligence" was still relatively rudimentary. For more than a decade, Siri has focused on handling some personal requests, and does not have the ability to track conversations, and misunderstandings are commonplace.
What is conversation tracking? For example, you ask ChatGPT what the weather is like in San Francisco, and then immediately ask it "How is New York?" ChatGPT will understand that you're asking about the weather in New York, but Siri won't.
There is no doubt that Siri has been overtaken by a large margin, and on a technical level, Siri has lagged behind. This prompted Apple, the tech giant, to undergo its biggest internal reshuffle in more than a decade.
According to reports, generative AI has now become a "pillar" project within Apple, a "a tent pole" project, a name unique to Apple itself, to organize Apple employees to participate in this once-in-a-decade innovation project.
What is certain is that Apple is going all out to catch up in this AI race.
Fourth, Siri will have a new set of generative AI systems, and the "hallucination" of large models on the end may become a challenge
We all understand Apple's determination, what will Siri become after the "brain change"?
At present, various reports believe that Apple will release an upgraded version of Siri at WWDC24 held on June 10, which is one of Apple's important AI moves.
The upgraded Siri will be more able to talk and will have more features. According to three people familiar with the matter, Siri's underlying technology will include a new generative AI system that will allow Siri to have the same true "chat" capabilities as ChatGPT, rather than just "one question and one answer".
Tom Gruber, co-founder of Siri, who worked at Apple until 2018, said that Siri has always wanted to have a conversational interface that can understand language and context, but this has been difficult to achieve.
According to three people familiar with the matter, Apple will not release a "ChatGPT-like" app to compete directly with ChatGPT, and Apple's focus is on giving Siri better access to the tasks at hand, such as setting a timer, creating a schedule, adding favorite items to your shopping list, and of course, the new Siri can also summarize text messages.
When Apple advertises the new Siri, it may focus more on its "privacy", because Siri will process all requests "on the device", that is, on the iPhone, instead of "going to the cloud".
This side can help Apple save a lot of money, you know, ChatGPT costs OpenAI 12 cents for every 1,000 words generated.
Of course, Apple's approach is not without its drawbacks, and one of the most common problems with the smaller model is the "hallucination" — making up answers that don't actually exist.
It is worth mentioning that Apple has also upgraded the memory to 8GB in advance on the iPhone 15 series, which is considered to be one of the preparations for the new Siri in advance. In addition, Apple is also negotiating with Google, OpenAI, Cohere and other companies about the licensing of AI large models, and is preparing in advance.
Fifth, more blockbuster generative AI functions will be implemented in system-level applications, and key functions still need to be coordinated by devices and clouds
In addition to Siri, according to foreign media AppleInsider, Apple is ready to apply real-time recording transcription and summary functions to a variety of products, and this function will be "system-level", that is, whether you are watching a video or having a meeting, you can record and summarize the key points.
Apple's Voice Memos app will be one of the first to receive this upgrade.
In addition, Apple's system-level note-taking app will also be upgraded to support basic text summarization and summarization functions, plus built-in audio recording and real-time transcription options, Apple's system-level note-taking app could become a key application for generative AI to land.
This is a boon for students taking notes in class and for business people attending meetings.
Apple wants to use AI to improve the efficiency of several of its core applications. Apple is currently testing these features for release in iOS 18 this year, and of course, they will also be available for macOS and iPadOS.
Transcription and summarization are reportedly just part of Apple's AI upgrades this year, and similar summarization features are expected to be applied to Safari and Messages apps.
Safari will support web summarization, while the Messages app will provide a "condensed version" of the message content.
While Apple plans to run AI on the device side, audio transcription and advanced AI summarization may still require cloud assistance for the time being.
6. Generative AI may subvert iOS and Android, and Apple's iPhone is afraid of "bricking"
Why is the arrival of new generative AI technology so "nervous" for Apple? According to two people familiar with the thinking of Apple executives, Apple executives are concerned that generative AI could threaten Apple's dominance in the global smartphone market, as the "new ecosystem" it could create could completely replace current smartphone operating systems, including iOS and Android.
It is worth mentioning that at present, there are mobile phone manufacturers like Honor in China who have proposed that the mobile phone operating system needs to be "redone" with AI, and it can be seen that the impact of generative AI on products such as smartphones must go deep into the bottom of the operating system. In this regard, the top tech giants "coincide".
AI based on new generative AI technologies, now also known as "agents", is actually an "AI application ecosystem" that can directly help users take taxis, arrange schedules, call various service interfaces, and call various large model capabilities, which will weaken the position of Apple's App Store, and users may no longer need to download apps to obtain services.
Apple's App Store has annual sales of about $24 billion, which is undoubtedly a big event for Apple.
In addition to worrying about the passivity of the big cake of the app store, Apple's other big concern is that if it can't develop its own AI system, the iPhone may lose its competitiveness and become a "dumb brickr" compared to other competitors' products.
At present, the annual sales of the iPhone exceed 200 billion US dollars, and Apple alone accounts for 85% of the profits of the global smartphone market.
7. Internal adjustment focuses on AI, and talent is still the "short board" of Apple's AI
This sense of urgency for AI to be left behind has made Apple move frequently for more than a year, such as canceling a $10 billion self-driving car project and transferring hundreds of engineers from the original project to the AI R&D team.
According to two people familiar with the matter, Apple is also exploring and developing servers that use iPhone and Mac chips, to put it simply, Apple wants to use its own self-developed chips as its own AI servers, and use A-series chips and M-series chips in servers, so that the cloud AI functions used in future products can run on servers that can truly "be controlled by Apple".
In this way, Apple's terminal products and cloud servers can also have better consistency at the hardware chip level.
Of course, doing so can also save Apple a lot of money.
According to Bloomberg, the M2 Ultra released by Apple may become Apple's first AI server chip.
Although Apple's current pace is a bit slow, Apple's advantages in doing AI are still obvious, first of all, Apple has more than 2 billion iPhones around the world, and Apple can apply new AI products to these devices.
At the same time, Apple has a strong chip team, which can solve the problem of computing power for AI functions.
Apple has reportedly spent the past decade working hard to develop AI-related technologies and build its own "comprehensive AI strategy," but Siri hasn't made any significant progress since its release.
John Burkey, who left for two years to start a generative AI company, Brighten.ai the company, said the Siri team didn't get the kind of attention and resources that other teams within Apple receive. "It's not in Apple's DNA, it's a blind spot." Burkey said.
For years, Apple has been recruiting AI talent and acquiring high-value startups in the AI field, but Apple is also facing a very serious AI brain drain.
According to reports, an important reason for the departure of AI talent may be Apple's too strict secrecy measures, Apple has published far fewer papers on AI than Google, Meta, Microsoft and other companies, Ruslan Salakhutdinov, a top AI researcher who left Apple in 2020 to return to Carnegie Mellon University, expressed his desire to return to academia at that time, he wanted to go to a research institute, to work in a more "open" place.
Although there has been a slight increase in the number of AI-related papers published by Apple in recent months, some big names in the AI field have questioned the value of these papers, arguing that these papers are more "face engineering" than research that can really allow Apple to land in products.
Conclusion: The software and hardware ecology is parallel, and Apple's AI big move is ready to go
Judging from the information from various sources, Apple's Siri's series of blockbuster generative AI upgrades, various generative AI features that will be added in iOS 18, and Apple's close negotiations with OpenAI, Google and other companies all show that Apple attaches great importance to AI, and we can see that Apple is doing AI on multiple fronts and all-round layout, from software, hardware, operating system to tripartite cooperation, Apple is in full swing to prepare.
Siri's "unstoppable" for many years may be about to become history, and Apple will reshape the intelligent voice assistant with large models like Google, Amazon and other tech giants.
Next, what kind of answer will Apple hand over at the WWDC in June, which is not only related to the development of Apple's own AI and the entire business disk, but also related to how the entire consumer electronics industry will apply AI in the future, and many companies are taking Apple as a benchmark.
Of course, many of the leading players in the field of smartphones in China have been in action for a long time, and various generative AI functions have already been implemented.