Apple's family system update: more AI, no more Hey Siri, watch major upgrade
Craig Federighi at Apple's Senior Vice President of Software Engineering at WWDC 2023.
On June 5, local time, according to past practice, Apple officially updated the software system of its various devices at the 2023 Worldwide Developers Conference (WWDC).
According to Apple, the iOS 17 system used by the iPhone has four major points: communication, sharing, intelligent input, new experience, and upgraded communication experience of phone, FaceTime and SMS; AirDrop makes sharing easier; Provide smarter input, improve the speed and accuracy of typing; Introducing the Journal and StandBy experiences, a new way to view information while the iPhone is being put down and charging; The Health app offers new mental health features.
It's worth noting that Apple canceled the "Hey Siri" voice assistant startup method, just say Siri to start.
Overall, Apple conservatively and cautiously added some AI features to iOS 17. Since the explosion of ChatGPT, public opinion has been expecting Apple to make moves in this competitive field. The famous analyst Guo Mingxi once said on Twitter that investors are more concerned about how Apple integrates ChatGPT-like AI functions than MR (mixed reality) products.
Unlike the conservative update of the Apple mobile system, the release of watchOS 10 confirmed previous rumors: it was one of the most important updates to the Apple Watch system ever, including an updated user interface, renewed focus on widgets, and updated native apps such as Compass, Mindfulness, Maps, and more.
On the computer side, Apple unveiled the latest version of its operating system, macOS Sonoma, which features support for desktop widgets, new game modes, and mobile aerial screensavers.
In terms of hardware, in addition to the much-watched MR headset Vision Pro, Apple also released the new 15-inch MacBook Air, launching the new Mac Studio and Mac Pro.
iOS 17, which has not changed much, adds a little AI functionality
Here are the key latest features available in iOS 17:
iOS 17 features at a glance.
Phone:
Get personalized contact posters set up for others to see when they call them, also available in the Contacts app, similar to a custom lock screen. Also available for third-party calling apps.
Get real-time voicemail with speech-to-text transcription before calling back, with the power of the neural engine, real-time voicemail transcription is processed on the device and completely confidential.
Bring FaceTime voicemail with quick video recording.
Personalized contact poster.
Facetime:
Audio and video messages are now supported.
・It can become more expressive with memes such as hearts, balloons, fireworks, laser beams, rain, etc. New effects can be activated with simple gestures, and third-party video calling apps can also be used.
Sms:
Swipe to restore any bubbles.
Audio messages are automatically transcribed so that users can read them immediately or listen to them later.
If the user shares the location, it will be updated in real time in the conversation.
· Check In is a new feature to notify families when it is safe to arrive. If no progress is made towards the destination, useful information is temporarily shared with the selected contacts, such as the device's location, battery level, and cell service status. Any information shared is end-to-end encrypted.
All emojis are now stickers. The all-new sticker experience adds the ability to create dynamic stickers by extracting subjects from photos, and effects can be added that help bring stickers to life.
AirDrop:
· NameDrop is a new way to share phone numbers and emails between nearby iPhones. Just put your iPhone together, or iPhone and Apple Watch together, and you can easily share contact information.
Shared media has also been redesigned so that long media transfers can continue over the Internet.
AutoCorrect:
Keyboard now leverages "Transformer Language Model" to improve predictions.
Sentence-level autocorrect can fix more types of grammar errors. New speech recognition models make dictation more accurate.
Personalized predictions based on word usage and usage. Users will now receive inline predictive text recommendations as they type, so adding an entire word or completing a sentence is as easy as hitting the space bar, making text entry faster than ever.
Journal:
Use machine learning on your device to create personalized recommendations for logs.
Photos, exercises, and interactions can be used as prompts. Prompts are intelligently generated based on recent activity, making it easy to start journaling.
Scheduled notifications can help develop the habit of keeping a journal.
Logs can be locked and end-to-end encrypted for privacy.
With the new Journaling Suggestions API, developers will be able to add journal suggestions to their applications.
Journal function.
Standby:
A new full-screen experience for displaying information in landscape mode. When the iPhone is charged on its side, the information is clear at a glance.
Data such as clocks, photos, widgets, weather, home controls, and third-party widgets can be displayed personalized.
· Siri uses a special landscape user interface in standby.
A red hue like the Apple Watch Ultra is used to display the clock at night.
On the iPhone 14 Pro series with a always-on display, features are always available.
StandBy function.
More:
No more saying Hey before using Siri to activate your voice.
· Safari adds better protection for private browsing. Advanced tracking and fingerprint protection upgrades to help prevent websites from tracking or identifying users' devices.
Health app offers new mental health features. Users can record their mood and momentary mood on a daily basis to see what causes their state of mind and easily access commonly used depression and anxiety assessments, as well as resources available in their area.
· The Screen Distance in Screen Time feature uses the TrueDepth camera to encourage users to move the device farther away after keeping it under 12 inches of their face for an extended period of time.
Maps added offline maps.
· AirTag can be shared with up to 5 other people, allowing friends and family to track an item in Find My.
· Apple Music introduces collaborative playlists to make listening to music with friends easier than ever.
Reminder to add grocery list, can automatically group added items, make shopping easier. Users can change how items are grouped, and the list remembers their preferences.
In Photos, People Album uses machine learning on the device to identify more photos of the user's favorite people, cats, and dogs.
Sensitive content warnings to let adult users know if they received a photo or video they might not want to see.
As in previous years, iOS 17 will be available this summer as a developer beta and public beta. iPhone users can update to the final version this fall.
"In iOS 17, we thought deeply about the features we rely on every day to make the iPhone more personal and intuitive." Craig Federighi, Apple's senior vice president of software engineering, said.
The most important update to the Apple Watch system
Although last year's debut on the Apple Watch Ultra added new features and a larger Apple Watch screen size, its user interface hasn't undergone a massive update in years. Meanwhile, many of the top third-party apps have exited the Apple Watch app store in recent years. App developers often say they realize they can engage users with notifications on the Apple Watch without having to maintain a completely native app for that purpose.
In response to this shift, Apple has doubled down on emphasizing that launching apps isn't necessarily the primary way to provide information and updates to Apple Watch wearers, and by introducing new widgets in watchOS 10, users can preview updates in a familiar format instead.
While the first Apple Watch had a widget system called Glances, the widget was removed in watchOS 3 as Apple pushed its native app agenda. The new widget is a combination of Glances and the widget introduced by iOS 14. In watchOS 10, users will be able to scroll through various widgets for common activities such as checking the weather or calendar, tracking events, checking stocks, etc.
A widget system called Glances.
To access the new watchOS 10 widget, simply turn the Digital Crown. Widgets like the weather or calendar appear in the screen's Smart Stack — a term Apple uses to explain the new interface. Previously, pressing the Digital Crown on the side would take the user to the home screen.
Machine learning will update widgets throughout the day as needed, allowing users to view morning weather, boarding passes, track medication times, or view sleep data, among other things. Apple explained that when a meeting or task appears, the calendar app and reminders will rearrange to the top, while running apps such as podcasts will also move up to be readily available.
Check the match score.
Third-party apps will also be able to use these widgets.
Meanwhile, pressing the side button will bring up the Control Center, while double-clicking the crown will return to any recently used apps.
In addition, Apple is updating its core smartwatch app, including the World Clock, with new background colors that reflect the time of day, as well as the ability to easily view different time zones of friends. The Activity app has been redesigned to include icons in the corners that you can tap to get weekly summary shares or rewards.
Apple also offers several new dials, including Snoopy and Woodstock, as well as a new color palette dial that changes over time with overlapping layers of color. Apple says that on Snoopy's face, the character can interact and play with the watch hands, react to local weather conditions, and even become active while the user is exercising.
Snoopy dial.
The Apple Watch also gets a new cycling workout feature, including a new workout view with new metrics, such as "Power Zones," support for "real-time activity" on iPhones, and the ability to connect to Bluetooth-enabled bike sensors. Other workout views are also optimized for iPhone displays, including heart rate zones, altitude, race routes, custom workouts, and a new cycling speed view that makes it easy for users to see data when they attach their phones to their bikes.
New cycling workout view.
The Compass app will now automatically generate two new stops – one that shows where there was last cell phone (last cellular connection location) and one that shows where emergency calls can be made on the route (last emergency call location). Users can also use altimeter data to access an altitude view, where they can view saved stops.
Apple says apps including weather, stocks, home, maps, messaging, world clocks and more can now take advantage of larger displays for easier navigation.
At the same time, developers will gain access to high-frequency motion data, which will help them use watchOS 10 to track actions such as swinging golf clubs or tennis rackets in their own apps.
Another change involves a new feature in mindfulness apps that lets users track their emotions and daily moods, and even scroll the digital crown to select feelings, which can then be tracked in the health app on iOS 17 or iPadOS 17, along with other health data.
Mindfulness can now record emotions.
The Apple Watch also has the same feature as iOS 17 to prevent children from getting too close to the screen.
Computer, iPad, and headset updates
Like the watch, macOS Sonoma, Apple's operating system, also emphasizes widget functionality. Widgets can be dragged from the notification center to the desktop, or if the device is on the same Wi-Fi network, it can be imported from the iPhone via Continuity.
macOS Sonoma also emphasizes widget functionality.
Macs also have a new "game mode" designed to prioritize gaming access to CPU and GPU resources and reduce latency for connected AirPods, Xbox, and PlayStation controllers. Apple says the mode works for all games on the operating system.
Video conferencing has also received some improvements, with Presenter Overlay that allows users to display things like slides behind them while their face is still in front of the camera. Users can also activate a set of meme AR effects during a call.
Sonoma will also provide an update for the browser Safari. Web apps can now run just like regular macOS apps, support notifications and be able to pin them to the Dock. The private browsing window is designed to provide more protection against cyberstalking. Apple also wants to make it easier to securely share keys and passwords through iCloud Keychain.
In addition to macOS-specific updates, Apple also said that several of the upgrades announced for iOS 17 will also apply to its computer operating system.
Sonoma features at a glance.
As for tablets, the new software update will get many of the same features as iOS 17, including automatic voice note transcription, expanded AirDrop functionality, and smarter text input autocorrection.
The iPad finally has the same personalized lock screen as the iPhone. Federigi also showed off new interactive widgets that can be placed on the home screen.
The iPad finally has the same personalized lock screen as the iPhone.
iPadOS 17 will also bring the Health app, as well as the ability to display real-time activity like iOS.
Federighi mentioned that the controversial Stage Manager will add flexibility in window size and positioning in iPadOS 17. This feature is designed for better multitasking on the iPad, but it has been mediocre since its debut last year, with occasional glitches.
On the Apple headphone side, the company announced several new enhancements, including adaptive audio, which intelligently blends the "noise cancellation" and "transparency" modes based on the surrounding environment.
When AirPods detect speech, the "conversation aware" feature will automatically activate transparency mode and lower the volume of music — just like Sony and other brands of headphones. Once you stop speaking, the audio will return to normal.
Apple also showed better background noise reduction to prevent the person on the other end of the call from hearing unwanted distractions.
A new AirPods voice feature is conceptually similar to Google's Clear Calling. "Clear Calling uses machine learning to identify, isolate, and eliminate distracting sounds, such as wind and traffic noise, while enhancing the caller's voice." Google once explained.
Hardware: The MacBook Air comes with a 15-inch option
Apple unveiled the new 15-inch MacBook Air at the conference, saying the device will be "the best 15-inch laptop in the world." It will come with Apple's M2 chip, eight-core CPU, ten-core GPU and 16-core neural engine. Memory can be configured up to 24GB and storage up to 2TB.
The new 15-inch MacBook Air.
The new model is 11.5mm thick, which Apple says is the thinnest 15-inch laptop in the world and will weigh more than 3 pounds. It has two USB-C Thunderbolt ports (which can support a 6K external display at 60Hz), a MagSafe charging connector, and a headphone jack, and is available in colors like Midnight, Starlight, Space Gray, and Silver.
The 15.3-inch LED-backlit IPS screen has 500 nits brightness and 2880x1864 resolution. Equipped with a 1080P FaceTime HD webcam, 3 microphones with "advanced beamforming algorithms", and 6 speakers with spatial audio and Dolby Atmos.
Apple says the new Air is 12 times faster than the fastest Intel MacBook Air model and has an 18-hour battery life.
Like previous MacBook Air models, the new 15-inch model will feature a fanless design. It currently runs macOS Ventura, and will be able to run macOS Sonoma this fall.
The new laptop represents a mid-range version of Apple, where there is a considerable price gap between the previous 13-inch MacBook and the larger, high-end models. The 15-inch Air will likely serve users who want a large screen but don't need the extra computing power (and cost) of the 14-inch and 16-inch Pro models.
The 15-inch size also serves as a middle ground between Apple's current 14-inch and 16-inch products. Users haven't had a striking 15-inch option for years, with a 14-inch screen often considered too small to comfortably use some creative programs, and the 16-inch Pro heavier to carry with peripherals and camera devices.
The new model starts at $1299 ($1199 for the education version) and will be available next week.
At the same time, Apple is adjusting the price of its smaller models. The 13-inch MacBook Air with the M2 now starts at $1099, while the base version of the M1 Air is still $999.
Apple also introduced the all-new Mac Studio and Mac Pro, two of the most powerful Macs ever made.
The new Mac Studio.
Equipped with the M2 Max and the all-new M2 Ultra, Mac Studio offers a huge performance boost and enhanced connectivity in a compact design. According to Apple, the Mac Studio is 6 times faster than the most powerful Intel-based 27-inch iMac and 3 times faster than the previous generation Mac Studio with M1 Ultra.
Apple unveiled the last chip beast of its M2 family, the M2 Ultra. With its support, a single device can run huge machine learning workloads such as Transformer. (01:19)
The Mac Pro now comes with the M2 Ultra, which is 3x faster than the previous generation of Intel-based models, with up to 192GB of unified memory.
Mac Studio and Mac Pro with M2 Ultra have more memory than state-of-the-art workstation graphics. The new Mac Pro completes the transition from Mac to Apple's own chips.
The Mac Pro now comes with the M2 Ultra.
Mac Studio and Mac Pro are available today and will be available June 13.