By | January 4, 2024
The Apple Vision Pro is on display during WWDC at Apple's headquarters in Cupertino, California

Days after reviewing Meta’s latest VR headset, the Mission 3, last fall, I was hospitalized with high blood pressure. Although the two events were unrelated, the latter changed my life quite a bit. In addition to having to pay even more attention to my eating habits, and go on a bunch of new medications, I was told to focus on getting regular cardio. These were familiar reminders: I have faced this path before.

When I told a friend that I needed to exercise more, she recommended a VR exercise app called Supernatural on the Quest. I laughed because I had already tried it, but this time I heard a review from someone I didn’t even know used VR. She got Mission 2 as a gift earlier in the year and have been using it ever since. So I joined again too.

I have been doing daily VR workouts for several months now. The Quest VR headset is my go-to training device. Meanwhile, downstairs, my oldest son is playing Beat Saber and The Walking Dead on Quest 2. For him, it’s his game console.

The future is now the present. Welcome to VR and AR, circa 2024. While Apple and the coming Vision Pro representing an exciting potential future for technology, Meta’s Quest and headsets are the already real and sometimes functional present.

But can these devices do even more? Can they transcend being game consoles, fitness equipment and experimental toys? Will VR and AR somehow grow up? It’s something I’ve seen slowly emerge during the pandemic years and the obsession with metaversebut in 2024, we may finally see products and apps that bring virtual augmented reality into the same functional universe as our phones and computers, perhaps finally leading to mass adoption like smartphones over a decade ago.

The Apple Vision Pro is on display during WWDC at Apple's headquarters in Cupertino, California

The Apple Vision Pro could be joined by a competing Samsung/Google headset next year. But we already know what Apples are.

John Kim/CNET

Apple Vision Pro: The device that can change the landscape

I caught a glimpse of the immediate future of AR and VR in 2023. Expected to go on sale in early 2024, Apple’s Vision Pro headset combines real-world camera feeds with incredibly high-fidelity virtual imagery. It’s a technology previously used by Meta and in high-end headsets such as Shadow XR-3 and XR-4but Apple has refined the interface with hand and eye tracking that, at least in early demos I’ve had, feels remarkably light and fluid.

Apple is also changing the pitch by replacing the terms “VR” and “AR” with its own phrase: “spatial computing.” Instead of focusing on gaming or fitness or even social connectivity to boot, the company’s marketing discusses how the Vision Pro can run all sorts of iOS apps and connect to Macs. Apple also emphasized a portable screen that is good enough to watch movies and photos in amazing quality. In my demos, that’s actually what impressed me the most.

The Vision Pro headset is Apple’s first major new product since the Apple Watch and is the most anticipated new hardware in the VR/AR space since original Oculus Rift. Despite its high price ($3,500) and likely limited availability, the Vision Pro could completely redefine virtual and augmented reality devices for the next decade. Or it could be yet another product in a landscape of forgotten headsets. Based on Apple’s history of successful products, the former seems more likely to me.

Will mixed reality app compatibility become a bigger trend after Vision Pro? It has to be. Will Apple’s controller-free design prompt other manufacturers to think about their own hand-and-eye interfaces, too? It seems likely. Meta already wants to sell its VR headsets without controls in the long run.

A VR headset, Quest 3, on a yellow table A VR headset, Quest 3, on a yellow table

Meta Quest 3 already does mixed reality for $500. Expect it to help explore new uses for the technology alongside Apple.

Scott Stein/CNET

Mixed reality at large

Everyday AR glasses, where virtual images and information appear in the lenses alongside the reality you see, are not yet a thing. There are many roadblocks: prescription concerns, safety, social acceptance, app compatibility, gesture interfaces, and just keeping them going for more than an hour on a charge.

Meanwhile, mixed reality VR has become the stand-in. Meta’s Quest headset, Apple’s Vision Pro, Varjo’s industrial headset and probably every VR headset going forward will have the capacity to blend virtual things with the real world using depth sensors and improved headset cameras.

The concept is still new, and so far limited. Meta’s Quest 3 relegating mixed reality to being a gimmick in some matches right now. Vision Pro, as we’ve seen so far, mostly uses mixed reality to float 2D displays over the real world.

By 2024, the door seems wide open for developers to figure out ways to use toolkits used today in phone and tablet-based AR apps like Pokemon Go. While we currently think of VR as “Hey, look at me, I’m in another world,” the experiential shift can quickly shift to pockets of immersion embedded in real-world environments. The Quest 3 is a device that I expect to lead that change.

An app menu in Apple's VisionOS. An app menu in Apple's VisionOS.

Apple has already opened up a ton of new potential apps for mixed reality. Samsung and Google could do the same.


Expect a new wave of apps

Apple won’t be the only new VR/AR platform in the picture. Samsung and Google and Qualcomm is set to debut its own mixed reality headset in 2024, likely offering another high-end alternative to the Vision Pro, but in a way that opens up for development via Google software. It sounds like a throwback to Google’s long dead Daydreamsbut it will likely mean opening up a whole new set of worlds: Apple’s App Store, and very likely Google Play as well.

For Apple, it starts with familiar 2D iOS apps translated into a mixed reality interface, but the possibilities beyond that could be huge. For example, Apple could add 3D movie support to Photos, or the Persona avatars to FaceTime. App developers already working on VisionOS software will have different ideas to solve than on the Quest’s hardware, which could also change the type of apps offered on each device. Games, for example, won’t be able to use standard VR controls on the Vision Pro: instead, we’ll have hand- and eye-tracking-powered apps.

Meta’s CTO, Andrew Bosworth, has already welcomed 2D apps to the Quest platform and told us that Google is moving Play to the Quest would be “trivial.” That hasn’t happened yet. Google could open up Android to mixed reality as Apple opens up iOS, both of these moves possibly opening the floodgates for developers who might not have been lured by more limited Quest, PlayStation, Vive, or SteamVR paths. VR’s pipeline of new games and apps has slowed in recent years, will Apple, Google and Samsung start a new movement?

Hands holding a pair of gray sunglasses with cameras in the corners of the frames, made by Meta and Ray-Ban Hands holding a pair of gray sunglasses with cameras in the corners of the frames, made by Meta and Ray-Ban

Meta’s glasses look normal, but in 2024 they will deal with AI via cameras and microphones.

Scott Stein/CNET

AI enters the mix

After a year completely defined by AI, it’s no surprise that AI would somehow find its way into VR and AR. For Meta in particular, it will be a big part of the 2024 plans. Metas Bosworth and CEO Mark Zuckerberg have already posted AI-infused plans for his metaverse dreams at the recent Meta Connect developer conference, and both Quest 3 and Meta’s Ray-Ban glasses has new control circuits with more robust AI processing capabilities.

Wearables like Humane’s Ai Pin and Meta’s new Ray-Ban glasses will both use cameras to fuel AI on the device. Camera-driven AI, which should work in a similar vein to how Google Lens “sees” the world interpreting the environment, objects and text, would make a lot of sense in camera-packed mixed reality headsets. There may be assistive devices, safety features, or just better ways for software to recognize the world and inject augmented reality into it.

The biggest immediate movement in this space may come in smart glasses instead of VR headsets.

TCL glasses on the face TCL glasses on the face

I tried AR glasses last year and many other times. They exist, but they are still experimental devices. Smarter everyday glasses are already here.

Scott Stein/CNET

What about glasses?

Qualcomm, and a number of other companies, are actively exploring ways to make glasses even smarter and more phone-connected. In the short term like glasses Meta’s latest pair of Ray-Bans focus on being camera enabled, microphone and speaker connected peripherals. In 2024, expect more layers of AI to help these devices listen and use camera data more intelligently, and perhaps act more like wearable assistants.

Glasses with a display are already available from a number of companies, including XReal, Lenovo and TCL. Expect more plug-in, monitor-on-your-face solutions in the form of glasses, especially as more devices become friendlier to USB-C connections and display technology improves all the time. I have already tried using one AR glasses-based “laptop”, so the idea is possible. Now they have to work better with regular glasses and feel less clumsy.

We may even see some AR glasses that aim to do (in smaller forms) what mixed reality VR headsets do. Yet the signs are clear that progress in that zone has slowed. Until mixed reality software improves and phone makers—namely Apple and Google—find ways to seamlessly wear tailgate glasses as everyday accessories, AR glasses won’t become a truly everyday thing for anyone.

Would I make my next pair of glasses a set of smart Ray-Bans? I’m already test-driving a prescription pair, and the experience is already profound at times. Smarter glasses will stake out the territory as VR headsets improve, and one day they’ll meet in the middle. It’s a day not this year, and it won’t be the year after. In 2024, all VR and AR need to solve is to find a few more ways to be really useful.

My family and I have already found a couple in fitness and gaming. I bet 2024 finds a few more.

#Practical #magic #big #thing..

Leave a Reply

Your email address will not be published. Required fields are marked *