A future without a smartphone, a world populated with holograms from the Web with which you interact thanks to two smart bracelets that seem to have come out of a Marvel movie. Facebook for the first time opens its Reality Labs and shows the futuristic project with which it intends to retire mice, keyboards and touch screens. “Every great technological revolution began by rethinking how we interact with digital, starting with Doug Engelbart’s invention of the mouse in 1968,” he says. Mike Schroepfer, Chief Technology Officer (Cto) of Facebook, the one who within the social network manages and designs all the innovations that his company uses or will employ in the future. “And we think it’s time to think about the next big turning point.”
Goodbye mice, keyboards and smartphones. Here are the Facebook smart bracelets
The new universe. The ‘human-computer interaction’ (Hci) starts from the idea that in a few years we will all wear glasses with lenses capable of superimposing three-dimensional images and information from the digital universe onto our surroundings. The so-called augmented reality (Ar), a dream that we could define as ancient by now, which even Google Glass tried to make present in 2013 with little luck, and on which they are working in a different form from Microsoft to Apple. The first with the Hololens viewer, which is already in its second version, the second with a device of the same type that should be announced in 2022 and a pair of glasses for 2025. Without forgetting other similar devices such as Magic Leap.
So imagine being able to view, through such devices, a screen wherever you are to surf the Net or watch series and films, to consult online the characteristics of the foods you are buying at the supermarket simply by looking at them or to compare their prices, to have them superimposed the directions to reach a certain place while you are on the street, to play video games that come to life in the environment that surrounds you. The problem is how to interact with all this having neither mouse nor keyboard, much less a touch screen. Hence the bracelets imagined by Facebook.
“For augmented reality to become truly ubiquitous, you need a technology that is so intuitive to use that it becomes an extension of our body,” he says. Michael Abrash, one of the leading researchers of Reality Labs. “So we had to invent a completely new type of interface that puts people at the center of this new experience.”
The bracelets in contact with the mind. Equipped with video cameras and managed by artificial intelligence, they will use sensors to translate the electrical impulses of motor neurons transmitted from the spinal cord to the wrist to the hand into commands. Commands that will then be used to control the digital elements in front of us. In this way, skipping the step given by a traditional keyboard or a touch screen, the level of interaction is on the one hand faster and on the other highly customizable and adaptable according to the situations.
According to Facebook, the signals that pass from the wrist are so clear that it is possible to deduce the slightest movements of the fingers, even if only by a millimeter. It might even be possible to perceive only the intention to move a finger and thanks to artificial intelligence to have predictive algorithms over time that anticipate our intentions making the bracelets even more precise.
A custom-made keyboard. Take for example the traditional qwerty keyboard invented 150 years ago. In its place we could have a virtual keyboard to display where we want it to learn and adapt over time to everyone’s unique typing style. In this way we would have a keyboard that slowly molds itself on us, faster than any mechanical typing interface and always available.
“What we’re trying to do with neural interfaces is to allow people direct control of the machine thanks to signals from the peripheral nervous system – the nerves outside the brain responsible for hand and finger movement,” he says. Thomas Reardon, director of the Neuromotor Interfaces division of Facebook Reality Labs. “It’s not about reading people’s minds, but about being able to translate many more thoughts into commands to be transferred to the digital world. It is a much faster way to act ”. In the advanced laboratories of the social network one goes so far as to argue that in this way the use of fingers could be restored, in the digital world at least, even to those who have not had them since birth or have lost them. Or that a person could add some to their virtual hand by managing them with the sole intention.
Holograms, Microsoft’s Hololens test
They call it “mixed reality”, it combines the real world with the virtual one. The only viewer on the market is that of the multinational founded by Bill Gates. Through the front cameras he is able to draw digital shapes that are connected in the environment that surrounds us. With applications ranging from work to play
The intelligent click. But in most cases, there will be only one command. A single interaction, the union of the thumb and forefinger, which will turn into what researchers call “intelligent click”. By interpreting the intentions of the wearer, bracelets should be able to transform this simple gesture into the necessary action to be performed in each different situation and context with a certain type of content. From grasping to moving, from opening to selecting. It involves a considerable calculation capacity. Microsoft also uses a similar system for its Hololens which, however, does not always work properly considering the possible variants.
The world of touch. But there is not only this, because the Facebook bracelets will even simulate the sense of touch. Nothing to do with the current haptic interfaces, those devices like the Sony PlayStation 5 controller capable of transmitting vibrations of different types according to what happens in the video game. “There is an enormous complexity in touch that is underestimated compared to sight and hearing,” he points out Nicholas Colonnese, part of the research team. “We have been studying and testing systems for reproducing it for four years with the idea of having a technology capable of encoding a great variety of sensations such as the hardness of an object, the texture of the surface, the temperature, the shape”.
From science fiction to reality, to create a new mass market. Apparently Facebook is seriously working to surpass what William Gibson imagined in novels like Johnny Mnemonic. The bracelets he is developing appear to be an advanced version of the virtual reality gloves that have populated books and films for the past three decades. And for sure they are just a piece of the puzzle that certainly also includes glasses and a software platform, an app store if you prefer, for digital services. Evolution of what Facebook already has for the Oculus headset.
In the background the idea of overcoming the smartphone, a market in which Facebook has never managed to enter with one of its devices, and which has now flexed almost everywhere since 2018 after a decade of breaking sales record after another.
The dangers. “We are at an early stage,” Mike Schroepfer puts his hands on. “There is no precise launch date, we are talking about years anyway. But before then we wanted to show what we are doing to also start a discussion on the advantages, the possible risks, the ethical implications “.
THE 6 PRIORITIES FOR THE ENVIRONMENT
2. Evaluate the side effects of innovation in advance
by Roberto Cingolani
The Facebook system, assuming that in practice it really manages to do what Reality Labs have hypothesized, facilitates contact with digital and eliminates a step as we have seen. In fact, it avoids learning to use the tools used today to manage our digital life. Yet learning to work with a tool, whatever it is, is part of a training path and simplifying something may not necessarily mean freeing more creativity or making people smarter but rather lead to a cognitive regression, as Roberto Cingolani pointed out. in the essay To prevent, written with Paolo Vineis and Luca Carra. Because, for every great innovation, there are always consequences and costs that are rarely evaluated. And this also applies to that “mixed reality”, where digital joins the physical world, which we don’t know what it could really bring. Especially if virtual objects were to have a presence and consistency that our senses risk confusing with real ones.
.