The new mixed reality headset that Apple has been working on for years is now “real” reality. The company is setting new standards with the device – but that comes at a price.
Apple’s headset for mixed reality is official and is called Vision Pro. It’s a whole new product category for the company, but could significantly disrupt the virtual and augmented reality market.
A separate operating system for the virtual home screen
At first glance, the user interface of the Apple Vision Pro is reminiscent of the iPhone or iPad. Many functions that are known from these devices are also available here. The proprietary visionOS operating system also enables video conferences with FaceTime in virtual meeting rooms. But how can other call participants see the wearer of the headset themselves? When setting up the Vision Pro, users have to film themselves once with the device, which then creates a digital persona. This avatar mimics all of the wearer’s facial and hand movements in the calls, and also has volume and depth to look as real as possible.
The Vision Pro is Apple’s first 3D camera. The headset offers new ways to interact with photos and videos. If you open the files, the room dims automatically. Self-taken panoramic photos appear life-size. The headset gives the 3D photos and videos a depth effect, so that the moments can almost be relived. The device also supports 3D films such as Avatar. In addition, users can watch videos with the headset as if they were on the big screen. You can adjust the window size as you like.
Apple makes it quite easy for developers to adapt their existing apps for visionOS. The new operating system has its own app store with its own applications. However, compatible iPhone and iPad apps are also available here. Similar to Face ID and Touch ID, the Vision Pro now has Optic ID, which scans the iris to unlock the device. This enables features like Apple Pay, App Store purchases and password access. For those who are critical of the eye tracking inside the headset, Apple reassures with the promise that apps and websites can’t track it. An additional level above registers the input and only then passes it on.
Apple has started a partnership with the Walt Disney Company, which is celebrating its 100th anniversary this year. Not only is Disney using Vision Pro headsets to create new 3D experiences, Disney+ will be available on the device from day one.
Can be operated without any accessories
Apple has developed a completely new control for the Vision Pro that does not require a controller or the like. Operation is exclusively via eyes, hands and voice. Cameras on the inside of the headset track eye movements and can thus identify what the user is looking at. These objects are then highlighted in the user interface. Pressing your index finger and thumb together then confirms the input. Finger swipe gestures are used for scrolling. For the search it is enough to look at a text field and dictate the input. A virtual keyboard is also available. Siri commands can also open apps like on the iPhone.
visionOS is fully 3D compatible. A 3D object attached to an e-mail can be simply pulled with a hand gesture and then viewed from all directions in augmented reality. But that’s not all: the system can also connect to other Apple devices. If you look at a MacBook, for example, the headset automatically projects the screen of the laptop. Apps can be started directly from there – without installation.
Despite this, the headset is compatible with Bluetooth hardware. For example, keyboards, trackpads and mice can be used with it. It can also be connected to gaming controllers such as Sony’s Dualsense, as the device also supports Apple Arcade. Although it has built-in speakers, AirPods can also be used for more private use.
TECHBOOK meint
“As the owner of the original Oculus Rift, I’m impressed with the engineering, software and looks that Apple has put into the Vision Pro. Judging by the presentation alone, this is easily the most mature product on the VR/AR market. Sure, things like SteamVR won’t run on it, which is why it’s hardly suitable, especially for gamers. But with its price, the device is more aimed at professional users who are used to completely different costs in this segment. I’m quite skeptical that Apple will really prevail in mixed reality. However, the company has proven time and again that it can conquer new product categories with iPhone, iPad, Apple Watch and AirPods.” – Adrian Mühlroth, TECHBOOK-Redakteur
Wearers get digital eyes
A special feature of the Vision Pro is a feature that Apple calls Eyesight. Because in order to be able to use the headset as naturally as possible in the real world, it is important to still be able to communicate with the wearer. So that he does not have to take it off, a curved OLED screen is installed in it, which shows the eyes outwards. Through a ventricular lens above the screen, the eyes are visible to the other person from every angle.
When used in augmented reality mode with apps or the like, the headset puts a slight shimmer over the eyes. However, when full virtual reality is activated, the eyes are covered with a veil. This allows other people to see when you are completely in the virtual interface. However, if a person approaches, the headset recognizes them and only makes their shape visible to the wearer.
Strong technology inside
Apple itself calls the Vision Pro a “spatial computer”. The headset has to calculate 23 million pixels, that’s more than 4K resolution per eye. This allows for crisp text to read and write, which has always been a weakness of VR/AR headsets.
For the necessary performance, the company relies on its own M2 chip, which is accompanied by a completely new chip: the R1. This is specialized in the processing of sensor data. It can process the data from 12 cameras, 5 sensors and 6 microphones without delay. Any delay, no matter how small, can contribute to discomfort in a mixed reality headset.
On the front sits a single, curved piece of glass that transitions into an aluminum frame that houses the button for the camera and Digital Crown. Under the glass are cameras for hand tracking and the 3D camera. On the inside are the two lenses, behind which are the two displays for the eyes. Glasses can be installed magnetically on the lenses, which correct individual visual defects. A “light shield” is used for sealing, which is available in different sizes. There are two audio pods that support 3D audio in the brackets. The pods should be able to adapt the sound to the room through “audio ray tracing”. The back part is made of braided, stretchable and breathable textile.
Continue reading: What is the difference between VR, AR and MR?
Who should buy the Apple Vision Pro?
With the Vision Pro, Apple is pursuing the goal of combining the real and the digital world. The company has developed a number of new technologies to make the transition as seamless as possible.
According to Apple, the main argument for using the Vision Pro is that users are no longer limited to a simple screen. Thanks to the 360-degree environment, the virtual interface offers almost unlimited screen space. Apps can be used anywhere, resized and placed anywhere. They have volume, react to light and cast a shadow. However, users see their real surroundings – the headset does not offer purely virtual, but augmented reality.
It is still capable of virtual reality. A Digital Crown, like the one found in the Apple Watch, lets users choose how much of their real world they want to see. With a twist, a kind of veil is placed over the field of vision and behind the content shown, which depends on the type and colors of the open windows. The degree of concealment can be freely selected.
Apple plans to place its headset as a new product category alongside the iPhone, iPad, Mac and Watch, but the mixed reality market is still young. Although the existing technologies are promising, they are still immature and expensive and therefore hardly suitable for mass production. So is the Apple Vision Pro, which at $3499 is no bargain for consumers. The company is at least trying to promote the device as a replacement for high-end televisions, computer monitors, audio systems, 3D cameras and more.
The New York Times reports that many Apple employees are skeptical about the new product. Some dropped out, while others were fired due to lack of progress. Internally, there is probably no consensus as to what purpose the headset should serve at all. This stands in stark contrast to Apple’s otherwise so goal-oriented product marketing – for example with the first iPhone: “an iPod, a telephone, an Internet communicator”.