Recommendation
Google AI Gemini's breakthrough, a new experience of real-time mobile interaction!
Core content:
1. Gemini adds real-time screen sharing function, accurately identifies screen content
2. Gemini turns on the camera to achieve direct interaction with the physical world
3. Google's leading position in the field of AI assistant is further consolidated
Yang Fangxian
Founder of 53AI/Most Valuable Expert of Tencent Cloud (TVP)
Google is still Google, and it is still the most powerful technology company on the planet.Gemini may only be slightly behind ChatGPT in terms of "topic popularity", but don't ignore its technical strength.Recently, a netizen accidentally discovered a new button - "Share Screen with Live" after activating the Gemini floating window .And this is exactly the new “AI real-time video feature” launched by Google for Gemini Live.Based on this,Users can let Gemini view the screen content in real time, or call up the phone camera screen.Gemini knows everything about the phone screen and the outside world
"Gemini, can you see my phone screen?"
"No problem, I can see the time is 8:50, the temperature is 71 degrees, and the date is March 23rd."
“Can you guess what Android system I have based on the screen content?”
"Hmm, it seems a little difficult to guess. It looks like a customized system."
“Which app on the screen is most relevant to my career?”
"Hmm, it seems to be Code Editor."
“Can you play the video for me?”
“Well, since we are having a real-time conversation, I can’t control the phone, but I can chat and brainstorm with you.”
Another feature launched simultaneously is the real-time video function.Open the full Gemini Live interface and start the video stream. There is a button in the lower right corner of the interface. Click it to switch to the front camera, so you can interact directly with the surrounding environment.It's like giving Gemini eyes so it can see the world.
After turning on this feature, Gemini can analyze the images captured by the mobile phone camera in real time and then answer the relevant questions you ask.
For example, in the demonstration video released by Google this month, someone wanted to use paint to color newly made glazed pottery, but was undecided about what color to choose. They could use this function to ask Gemini for help.
In response, Google said it would "gradually push it to Gemini premium subscribers as part of the Google One AI premium plan" in late March."Project Astra" Google's long-planned AI assistant
“This vision has been in my mind for quite some time.”Demis Hassabis, head of Google DeepMind and leader of Google AI Efforts, mentioned this at the Google I/O conference in March 2024.Demis Hassabis showed off a very early version of what he hopes will be a general-purpose assistant, which Google calls "Project Astra."It’s a real-time, multimodal AI assistant that sees the world, knows what things are and where you put them, and can answer questions or help you do almost anything."Gemini Live Shared Screen" is the first appearance of "Project Astra".It is also a trailer for a certain product "form" in 2025, the year of AI Agent.When AI is "equipped with eyes" to observe the real world in real time and interact with you and your phone.What kind of sparks will AI create when it collides with humans?The launch of this new feature by Google further consolidates its leading position in the field of AI assistants.
