When wearing an Oculus Quest 2 into the Meta VR social platform Horizon Worlds, you need to create an avatar to represent yourself, but it doesn’t look like you in any way, because it doesn’t have a lower body, like Hogwarts Ghost Walks, and recently floated into a Super Bowl commercial.
The same is true of Microsoft next door. The avatar of the Metaverse entrance Mesh for Teams also has no legs. Then you blurt out: why no legs?
How hard is it to add legs?
Andrew Bosworth is the VP of Meta Reality Labs, where he developed the habit of instant Q&A via Instagram while leading the XR division. In the February 10 Q&A, he admitted that adding legs is difficult.
If you want to present your legs in a realistic way in VR, you first need to know what the legs are doing in reality, and then infer what the legs should do in VR. The difficulty is that while the Quest 2 can track the head and hands and estimate the position of the arms and chest, it doesn’t know where the legs are.
Quest 2 is equipped with 4 wide-angle grayscale lenses, 2 in the upper corners and 2 in the lower front. If we are looking forward, these shots usually see the arms, and if we are sitting or looking down, the shots can also see more of the body. However, the tracking range is limited, and sometimes “obstacles” such as the abdomen may block the camera; sometimes tilt or turn the head, and the camera below cannot see the legs. Meta also wanted to make the device smaller, making it harder for the camera to see the legs.
（Source：Oculus Quest 2）
And what about leg device sensing? Theoretically possible, but there are few commercially available sensors and controllers for the legs.
HTC sells the Vive VR device for business users, and also has trackers that attach to limbs or objects such as tennis rackets. However, the tracker can only be connected to a computer device, and a locator needs to be set up. When it comes to the locator, the core reason for “no legs” comes out, that is, the positioning method of Quest 2.
VR positioning technology can be divided into two categories, one is outside-in, and the other is inside-out. Outside-in needs to place multiple locators in advance. The locators will emit lasers, infrared rays, visible light, etc. to cover the space between the locators to establish three-dimensional spatial information, and determine the user’s position and movement direction through triangulation.
（Sourec：WAREABLE）
Quest 2 is an inside-out, no need to set up a positioning device, only the VR equipment is installed with the lens, and the equipment can detect the changes in the external environment by itself, and then use the SLAM algorithm to calculate the spatial position and moving direction of the lens.
To put it simply, inside-out positioning does not require a locator, so it is free to use space and is more suitable for daily entertainment and movement; outside-in can provide more accurate and wider tracking effects, and full-body motion capture generally relies on outside-in tracking settings achieved.
Since full-body tracking is currently unavailable, Meta and Microsoft “cut off” the waist down.
The positioning method is related to the usage habits and the company’s strategic direction – the more convenient the technology development, the better, and consumers do not necessarily care what technology it is. The Quest 2, which starts at $299, lags behind in accuracy and latency, but its positioning is friendly enough for the average user.
Running a VR community has always been the goal of many VR manufacturers. Meta certainly wants to bring VR to the mass consumer market. While more realistic physical representations will certainly drive more interest in VR, it could also make the user experience more cumbersome and costly, at least in the short term. The so-called “two rights harm each other, whichever is lesser”, a lower threshold for use is of great significance to the popularization of VR.
Timoni West, VP of Augmented and Virtual Reality at Unity, has come up with a solution by collecting large amounts of walking data and using AI to predict leg movements based on head movements detected by the device. AI may not be able to accurately predict every movement of the user, which will cause AI to reproduce the movements unnaturally; the premise of “collecting a large amount of walking pattern data” is not easy. Creating virtual models of human motion may no longer be difficult.
In short, it is virtually impossible to require a consumer product all-in-one VR device to perfectly track the legs without having many sensors on or around it. As O’Brien, general manager of HTC America, said:
The average person has always pursued a full-body immersive experience, reproducing every movement of the body with an all-in-one device. But we can’t do it yet.
Do you want to dance?
In September 2019, Meta announced that the Oculus Quest uses deep neural networks to accurately track hands. Wearing the Quest 2, we can simply operate the main interface by waving our hands, or hold the controller to kill the Quartet in the game. In Horizon Worlds, floating “legless monsters” can hang out, meet friends, shoot zombies, or create worlds.
Why is accurate hand tracking important? Because of unlocking a series of new experiences – pausing the movie with gestures, social games are more natural expressions, waving to say hello, guessing, shaking hands, taking, high-fives, etc. as in reality. In contrast, leg tracking seems less necessary if gaming uses such as dancing are not required. Even without leg tracking, the software can simulate walking, squatting, and more.
In order to make walking more realistic, some researchers have launched a foot VR force feedback scheme. For example, the somatosensory sensation when walking includes changes in the visual scene, feedback from stepping on the ground, and the somatosensory sensation of moving limbs. Therefore, in theory, by stimulating and simulating these somatosensory and visual effects, a realistic walking experience can also be simulated. If you just don’t like it, it’s enough to wear a “prosthetic leg”. Andrew Bosworth thinks so:
We can make prosthetic legs for the avatar, so people can see the full image, and no one will know the difference.
Of course, Andrew Bosworth also admits that artificial legs cannot replace real legs. The prosthetic leg just makes the floating avatar look more natural when hanging out, chatting and other simple VR social experiences, but without the flexible legs, it always feels like something is missing. For a more immersive and realistic VR experience, we always need legs, which is represented by the VR community platform VRChat.
A major feature of VRChat is that there are no restrictions on virtual characters. The well-known MMD model of the second-generation rice can be directly converted into VRChat characters, and Kirby, SpongeBob, and Hatsune Miku appear together. Not only that, but more and more VR dancers come to hip-hop battles with motion trackers strapped to their bodies.
For full-body dance in VR, South Korean VRChat dancer Makoto’s trackers are exceptional: an HTC Vive with wireless add-ons, an Index controller on the wrist, and a waist and foot Vive Tracker.
（Source：MAKOTO）
VR enthusiasts are willing to pay a lot of money and energy for full-body tracking, which means that the threshold for having “real legs” is still high. But in the longer term, communities are formed around these events. The world‘s first VR dance community @IDA_officialVR is holding a series of dance competitions at VRChat. VR truly connects people who may never meet. We still look forward to the day when the technology is better and cheaper, and VR can provide more of a life-like experience.
Perhaps it can be seen from the question of “why don’t have legs” that compared to Zuckerberg’s metaverse, VR technology is still some distance away, and Horizon Worlds is not a complete metaverse. Then again, maybe not everyone likes having legs, and not everyone wants an avatar that reflects everyday life. At least, in the VR world accessible to everyone in the future, let the “floating legless monster” become one of the choices rather than regrets.
(This article is reproduced with permission from Ai Faner; source of the first image: Horizon Worlds)