Google shows Astra, a rival to GPT-4 that sees the real world through the cell phone camera

During the traditional developer event Google I/O we know the Project Astra, one new artificial intelligence which is being developed by the company. Defined as a Multimodal AI with support for creating different types of content, understand how it works in practice and how the new product can interact with the real world, providing answers to everyday questions.

How Astra works

Easily defined as an AI agent that should be integrated into the application Gemini for smartphones Android, Project Astra “sees” using the smartphone’s rear camera and sends real-time responses to its user. It's as if your smartphone came to life so you can see everything around you.

During today's Google event, we saw Half Hassabis testing AI in practice — and the head of Google DeepMind and leader of the AI ​​efforts promises it has not been manipulated. We were transported to a UK office and, as soon as she saw a speaker, the user asked the AI ​​to identify one of its components, a tweeter.

Another interesting demonstration was reading programming codes on a computer screen. O Project Astra not only was it able to identify that the monitor was displaying codes, but also indicate what they would do when they were running.

Finally, when the smartphone was brought to the office window and the user asked which neighborhood the office was located in, a response was sent in less than five seconds.

O Google defined the Project Astra , the “a useful AI agent for every day”, and that the development was carried out over the last five years to deliver a tool that really makes a difference in users' daily lives.

While we've made incredible progress in developing AI systems that can understand multimodal information, reducing response time for something conversational is a difficult engineering challenge. Over the past few years, we've been working to improve how our models perceive, reason, and talk to make the pace and quality of interaction more natural.

Google about Project Astra

Memory Resource

What caught the most attention in the Project Astra it was the possibility for him to memorize everything seen by the camera. When the user asked where her glasses were, the AI ​​agent quickly answered the exact location, which was next to a red apple.

When put to work on an undisclosed smart glasses, the Project Astra It also impressed by making optimizations that make everyday life easier for many people. The conversational style of Google's AI agent is also very light, showing that the company is committed to developing AIs that do not leave naturalness aside, even though they do not exist in real life.

By leveraging our core speech models, we also improve how they sound, giving agents a wider range of intonations. These agents can better understand the context in which they are being used and respond quickly in the conversation. With technologies like this, it's easy to imagine a future where people could have a specialized AI assistant at their side, via a phone or glasses.

Google about Project Astra

Check out the example shown today at the Google news event:

Availability

According to its own data Google, Project Astra will be integrated into the Google Gemini, replacing the famous Google Assistant, later this year. Early adopters are likely to see this in mid-September, the time of year when the Google launches its new smartphones pixel and will also release the Android 15 for smartphone manufacturers around the world.

The company's idea is to develop small features that, together, form a complete option for its users. O Project Astra is just a portion of several new features presented in the Google I / O 2024, but it was certainly clear that the company, which mentioned the term “AI” 121 times in the two-hour event, will invest more and more in this market.

Google Assistant gains new use

To date, other personal assistants like Amazon Alexa e Crab are very focused on sending simpler responses and user-uploaded files. From the moment Project Astra is integrated with the Google Assistant, present on several Android devices and even available to those with Apple devices, this will certainly usher in a new era.

The big question is when this should happen and especially how it will work in practice. Because it is still in development, the AI ​​agent must undergo some modifications before being released to everyone, but the idea is interesting and now has our attention.

What did you think of Project Astra? tell us Comment!

See also other features

Gemini Advanced or ChatGPT Plus? See which AI is the best

Google announces Gemini 1.5 with 87% better performance

reviewed by Glaucon Vital in 14 / 5 / 24.