At today's Made By Google live event, Google announced that it will be upgrading its AI for Google Assistant with a new version of Google Assistant powered by generative AI technology called Bard Assistant. This is essentially a combination of Google Assistant and Bard on mobile devices, and the new Assistant will be able to handle a wider range of problems and tasks, from previous simple requests such as "What's the weather?" "Set an alarm" or "Text Jenny," to smarter responses now powered by Google's Bard AI.
This means that Bard Assistant can dive deep into your own Google apps, such as Gmail and Google Drive, to provide personalized responses, provided you voluntarily choose to allow it. This means you can ask questions to Bard's assistant like "Help me check important messages I missed this week," and the digital assistant can figure out what you need to know about.
This feature builds on an update released by Bard in mid-September that allows AI chatbots to integrate with Google's own apps and services through "Bard extensions," including Gmail, Docs, Drive, Maps, YouTube, and Google's flight and hotel services. Users who have already allowed Bard access to their Gmail, Drive, and Docs do not need to reauthorize when using the assistant app, but users who have not yet tried to extend the functionality need to authorize Bard in the assistant app to respond to personal queries.
In addition to finding content in your inbox, Google says the extensions can also be used for personal tasks such as travel planning, creating shopping lists or writing headlines for social media. By launching this new experiment, Google plans to study how people use Google Assistant with Bard before making the feature widely available to general users of Android and iOS.
Since Bard is now also available on mobile devices, users can interact with it in a variety of ways. Sissie Hsiao, vice president of Google Bard and Assistant, explained of the new feature: "It can hear sound through the microphone. It can talk to you through the voice output. It can see things through the camera. It can even take action to help you. Of course, it is located on the device that you carry with you, which is your phone. ”
The expansion was seen by Google executives as a major leap forward for the company's digital assistant, which previously limited its functionality to more basic tasks.
Over the past seven years, Google Assistant has been helping millions of people complete tasks through natural and conversational means. For example, with a simple 'Hey Google,' you can set an alarm, check the weather, or make a quick call. Now, with generative AI, we have the opportunity to provide a smarter, more personalized, and more intuitive digital assistant. We think this should be more than just sound. ”
In fact, there are three ways users can interact with Google Assistant combined with Bard. They can ask questions and follow questions with their voice, type queries, or use the camera with Bard's Google Lens integration. The latter allows users to take photos or upload pictures to illustrate their queries.
Hsiao says people use the feature in a number of unique ways, such as taking photos of their clothes and shoes and then asking Bard how to match them, or taking screenshots of the app and asking Bard to write the code framework.
She explains: "We wanted Bard to be able to operate in multiple modes. It can be seen. It can be heard. It can talk to you. ”
In addition, on Pixel devices and some Samsung phones, you can long press the power button or the home button to pop up a pop-up floating window respectively, providing conversational overlays on the page you are viewing, allowing Bard to respond to what you see on your screen. For example, you can pop Bard on a picture of a hotel and ask if you can book the hotel this weekend.
With Bard's integration with Google Assistant, it is no longer subject to any restrictions from the web version, which means that if there is concern about hallucinating AI, it can now double-check the answers as well. This is the problem faced by modern AIs because they construct incorrect answers based on false information. This feature was also available in mid-September.
Google says the Assistant with Bard will initially be available in some limited markets, not just the English-speaking market. However, it has not yet been determined which markets or languages will get this update first. In the coming months, Google will roll out among iOS and Android mobile users, and then explore the possibility of bringing the upgraded assistant feature to other platforms.