Google I/O 2024 Live Blog: Gemini AI Features Are Coming to Many Apps

Google has wrapped up its AI-heavy Google IO keynote presentation. In case you were wondering, Sundar Pichai revealed that Google mentioned AI 121 times today. That's a decline from the 140-plus times it mentioned AI during last year's I/O. 

Nearly two hours into Google I/O, Google offered up a segment on how it's responsibly using and training AI -- better late than never. It's actually using AI to train AI, in a process called Red Teaming. Google is also calling out the possibility that Imagen and other tools could be used to spread misinformation. It's spreading its SynthID watermarking capability to multimodality text and generated video, and will open-source SynthID. On top of that, Google will soon release a Responsible Generative AI toolkit for developers to "build AI responsibly."

Yet the company didn't touch on what data its Gemini tools -- which most of Google's AI products use today -- are trained on. Google's responsibility is focused on what's created using its tools, not what creations are used by its tools.

A new feature that's in testing uses Gemini to warn you about potential scam calls. On stage, Google showed how it listens in to your calls. In the demo, when the caller asked to shadily transfer money to a new account, an alert popped up on screen that the call was likely a scam. Given that analysis stays on the device, it's more secure, and could protect people when scammers use phishing tactics like scaring and pressuring them to choose quickly. This builds on Google I/O presentations of yesteryear of tech that acts like a concierge to screen calls and make restaurant reservations. 

Looking for what'll be new in Android 15? That won't be coming today. Despite all the Gemini-in-Android talk during the keynote presentation, Android 15 is still coming and Google will indeed talk about it at IO -- just not today. Google will instead be detailing its next update for its mobile software tomorrow. Stay tuned. 

Google's Circle to Search feature is readily available on 100 million Android devices. Now Google says it plans to double the amount of devices that can access the feature to 200 million by the end of the year. 

There's a new Slackbot in town. The Gemini-powered "virtual teammate," called Chip, can be integrated into your work chat to check on things and surface topics buried in months and years of conversations. On stage, Google demonstrated how someone asked Chip about the progress of a work project, and the chatbot found who talked about it last and intuited its status. This could be helpful, as work chat search functions are iffy at best. Let's just hope it doesn't hallucinate that you were responsible for a task that hadn't actually been assigned... and then go tell your boss.

There's a lot of Gemini news at Google I/O, and that includes how its presence on Android will change. Gemini will be able to understand more context about the app you're using, meaning it can answer questions about the video you're watching on YouTube or generate images that can be dragged and dropped directly into Google Messages or Gmail. It's a sign that Google wants AI to play a bigger role in Android. 

Until this point, a lot of the generative AI features we've seen on phones have been targeted at very specific tasks, like editing an image or translating a conversation. But this Gemini update points to a future in which AI changes the way we use our phones on a more fundamental level. Read more about it in my full story. 

Google will leverage Gemini Nano to bring richer and clearer image descriptions to its TalkBack screen reader. TalkBack reads aloud what's on a screen and helps blind and low-vision users navigate their devices using custom gestures and voice commands. Now Gemini can offer more details about what's being shown in an image or an online product listing. In a blog post, Google shared an example showing a dress, for which TalkBack generated the description, "A close-up of a black and white gingham dress. The dress is short, with a collar and long sleeves. It is tied at the waist with a big bow."

Google Gemini is now very conversational. With Gemini Live, you can have a natural-sounding conversation with Gemini, even interrupting it midsentence. Google says Gemini will also be more capable, allowing users to post massive amounts of text for analysis. You can read my full story about a wide range of Gemini announcements. 

Suggestions for email replies have been a part of Gmail, but now Google has revealed that it'll soon be able to reply with customized answers based on the context it gleaned from the email thread. 

You won't have to wait long to try Google's latest AI-based Search. The company says it'll be rolling out some of these new features "in the coming weeks," with eager users able to access them first by opting into Search Labs.

Forty-five minutes in and we've heard "AI" dozens of times, but there's been nary a mention of how Google is building these tools and models responsibly. Where will the new tool Veo source the material it's basing new generated videos on? What about the audio tools? And how will the artists who made the content that's fed into models be compensated? Full details can wait until after the keynote presentation, but it's telling that Google hasn't even had a throwaway line about using AI responsibly.

AI has dominated Google I/O during the first 45 minutes, and now we're focusing on how AI and Gemini will be impacting Search. Google says it's adding "multistep reasoning" into search so you can submit multiple questions in one search and have AI put everything together. 

OpenAI's ChatGPT-4o made waves on Monday for its AI assistant that seemed to eerily mimic Scarlett Johansson in the movie Her. Google's latest update to its Project Astra appears to be its answer for a multimodal version of Gemini that can be helpful. In a demo video, the company shows the AI recognizing content in a video and accurately answering questions and identifying subjects. 

Google says the Gemini app will be getting some of these experiences later this year. 

We knew Google was going to lean into AI at I/O, but we're minutes into the keynote presentation and its generative AI tool Gemini has been demonstrated to give nuanced and context-sensitive responses to everything from searches to responding to questions so naturally it looks like a conversation. From here, everything we see today could have AI involved in the output.

Google is bringing its Gemini AI to its Workspace apps like Gmail and Google Meet, where it can perform tasks like going through a series of emails from a school and summarizing the key points to save you time and catch you up.  

Google search in the United States is going to be getting more AI-driven results. The new feature is coming to the US first, but will be rolling out to other countries "soon." 

CEO Sundar Pichai has hit the stage and Google I/O is officially underway. Pichai describes I/O as "Google's version of the Eras Tour" and says the company is now in its "Gemini Era." Get ready for a lot of AI talk. 

We're taking our seats at Google I/O 2024 and as we wait for the conference to start, a DJ is playing music ahead of the event.

Google announced the Pixel 8A last week, a follow-up to last year's Pixel 7A. But there's a problem. The Pixel 8A costs $499 and Google still sells the 1-year-old Pixel 7A for $499. Please don't buy a Pixel 7A unless it's discounted. My guess is that Google will eventually drop the price of the Pixel 7A -- either during Google I/O this week or when the Pixel 8A is available in stores. But if you were planning on dropping $500 on a Pixel 7A and didn't know about the Pixel 8A, then congrats: You got an easy upgrade to something newer and shinier without having to spend a penny more.

Google hasn't gone big on new hardware at IO in recent years, but it does occasionally use its flagship developer conference to tease its hardware plans more officially. Google's Pixel division is notorious for leaks, and just the other day hands-on images and details of what appears to be the forthcoming Pixel 9, Pixel 9 Pro and Pixel 9 Pro XL appeared on Russian website Rozetked. According to the outlet the Pixel 9 will sport a 6.24-inch display, the 9 Pro will bump that to 6.34-inches and the 9 Pro XL will jump up to 6.73-inches. All three will be AMOLED displays and support 120Hz refresh rates, as well as rock some new rear camera designs.

Will Google make the phones official on Tuesday? We'll find out soon enough. 

It's been a year since Google I/O 2023 and the debut of the Pixel Tablet, a digital slate that, with its quad-speaker charging dock, is intended to cater to folks who want something more advanced than an Echo Show in their home. We gave the tablet top marks, and though there's been no hint that Google will follow up with another tablet at Google I/O 2024, perhaps there's a software update in store.

Google I/O 2024 is providing attendees with a breakfast before the conference kicks off. This includes matcha croissants and muffins.

Google could use the week of I/O to unveil some digital accessibility updates, since Global Accessibility Awareness Day this year falls two days later, on May 16. The company has recently rolled out features like Guided Frame, which helps blind and low-vision Pixel users take selfies. It's also launched apps like Magnifier, which makes it easier to see small text and objects, and added AI image descriptions to its Lookout app, which helps people who are blind or low-vision identify objects and read documents. It's likely Google will continue to use AI to boost digital accessibility across more products and services, showcasing the "good" the rapidly developing tech can usher in.

While the 2021 release of Wear OS 3 -- a combined effort between Google and Samsung -- promised a new era for non-Apple smartwatches, updates have been somewhat sparse with Wear OS 4 coming last year alongside the Samsung Galaxy Watch 6 featuring a handful of improvements. But a rumor says that the upcoming Galaxy Watch 7's chip is running software referred to as Wear OS 5, according to 9to5Google. It's unclear if it'll properly debut at Google I/O 2024, as Google sometimes waits to reveal Wear OS news until the smartwatches running them debut.

At Google I/O 2024, we expect to see a lot more about Android 15, the next big software update for Android phones. It's already available for developers and early adopters with Pixel phones to try out in beta, and we've seen its improvements on productivity, privacy and security, including archiving apps to free up space on your phone and more support for satellite connectivity. Google I/O is when we'll hear about many other ways the Android experience will change.

Google has invested heavily in Gemini, its generative AI platform -- and it will hardly be surprising if Gemini features in a big way at Google I/O. Google could even go as far as showing off an entirely new version of Gemini. The search giant will seek to prove its superiority over competitors like Microsoft and OpenAI, which on Monday unveiled a major update to GPT-4 that it showed can detect human emotion and also offer information in real time. But if a brand new Gemini doesn't make a debut at I/O, at the very least we can probably expect some generative AI updates for major Google products such as Android, Maps or Search.

Software is usually the focus of Google I/O, but Google surprised us with a few hardware teases in recent years, such as the Pixel Fold last year and the Pixel Watch in 2022. But given the company's laser focus on AI over the past year, I'd be surprised if it carved out time for new hardware announcements, especially since it introduced the Pixel 8A just before I/O. Instead, we'll probably hear more about AI updates to services like Search, Maps, Gmail, Docs and the Gemini virtual assistant. Google usually saves major Pixel product announcements for the fall.

Last year, Google provided a glimpse into how generative AI could surface on Android phones by announcing features such as Magic Compose (for rewriting text messages in a different tone) and AI wallpapers. A lot has changed since then; from Circle to Search to new translation features and photo editing tools like Magic Editor, our phones have gotten a big dose of AI over the past year. I wonder if Google will introduce new features, or perhaps announce expansions for current tools like Circle to Search

Google Assistant was noticeably absent from last year's Google I/O, which I found odd, considering so much of the event was focused on AI. In 2024, Google has seemingly been focusing its resources on its new Gemini assistant, which makes me wonder whether it will eventually phase out Assistant altogether. Either way, I'll be looking for clues about Google Assistant's future based on how the company positions it – or ignores it – at Google I/O. 

Google's affordable-ish Pixel Tablet, which debuted at last year's Google I/O, was the company's return to the large-screen format. After ceding ground to Apple's iPad, Samsung was largely holding down the premium Android tablet fort, doing the best it could without Google's full support. But Google's re-entry to the tablet market put some much needed fuel in Android's smoldering tablet ecosystem. Now, ideally, Google would want to keep that momentum going with a followup to last year's Pixel Tablet with a Pixel Tablet 2, right? It does seem, however, that we won't be seeing a new Pixel Tablet at I/O this year. 

Ahead of the Pixel Tablet's launch last year, there was a steady stream of rumors flowing into the mouths of Android's acolytes. Leak upon leak all but suggested a tablet was on its way. However, there hasn't been nearly the same flood of rumors for the Pixel Tablet 2 as there was for the first. Other than a codename mentioned in a beta Android 14 update, we haven't seen any design leaks or third-party case designs hit the web. Either Google's doing an unusually good job of keeping the Pixel Tablet 2 under wraps, or it doesn't have anything to show at this year's I/O.