Hands On With Google’s Gemini-Powered Smart Glasses, Android XR, and Project Moohan Headset

Of course, you can work in a mixed reality environment using a connected Bluetooth keyboard and mouse, and you can place yourself in an immersive environment if you want to focus, or turn on see-through mode to make sure your colleagues don’t notice Get your work done while taking photos and giggling while wearing ridiculous headphones. It’s unclear whether you’ll be able to connect headphones to your laptop to bring your work into mixed reality (a feature available on Apple Vision Pro).

Gemini in XR

Clicking on the side of the headset brings up an app launcher, and you can toggle it here if you want the Gemini to stay “on” all the time. Once opened, there will be an icon at the top of the virtual space so you know everything you say and see is being recorded by Gemini.

In see-through mode, you can walk up to an object and ask the Gemini about it — a Google employee demonstrating the headset (before I tried it) walked up to another guy wearing a FC Barcelona jersey and asked Gemini Find “Rankings for this”. Gemini quickly registered the team name and pulled up search results, which included league standings and scores from recent games.

You can ask Gemini similar questions and it will answer through visual results displayed in the headset. I asked it to “Take me to Peru,” and it opened a 3D version of Google Maps. I was able to move around and in the center of Lima, and in a city where the map already has extensive 3D models, you can explore areas in greater detail. You can talk to Geminis constantly during these experiences, so I asked some questions like when is the best time to visit and got prompt answers.

In another example, I took a virtual tour of the interior of a restaurant in New York City. Google says it can use artificial intelligence to stitch together and display images from inside the venue to make you feel like you’re actually there. It was pretty well done and I asked Gemini if ​​the place took reservations without naming it as I was eyeing the name of the restaurant. it Do Reservations are accepted but Gemini couldn’t actually book it for me. (This integration may come later.)

Next, I watched some videos on YouTube where the 2D content looked sharp and colorful. The stereoscopic content was even better; my senses were enveloped. I saw some hikers walking along a trail and I asked Gemini where all this was and it said, “New Zealand.” I couldn’t verify this, but it seemed like the correct answer. I watched some more spatial 2D videos playback because the virtual player added depth and layering to make them feel like 3D. I jumped into the Google TV app and enabled “Cinema Mode” to launch a virtual theater for watching movies and shows, just like on other VR headsets.

Circle searcha feature Google launched earlier this year on Android phonealso available in Android XR. Simply walk up to an object near you, press the button on the top of the headset, then pinch and draw a circle around the object you want to learn more about. You’ll see a Google search page with results.

smart glasses

The Moohan project feels like Google and Samsung are playing catch-up with the rest of the VR market, though the Gemini integration brings a unique dimension to their efforts. However, I admit I’m more excited to try smart glasses, and Gemini feels it might be more helpful. They did not disappoint. I walked into another room and there were several pairs of glasses in front of me. Some are sunglasses and some have clear lenses. Just like headphones, you can ask them to load your prescriptions. Google did not provide a name for the prototype glasses.