Whether in the Metaverse or on Apple’s Facetime calls, avatars still look astonishingly bad. The startup Meshcapade is working on avatars that not only resemble us, but also move like us.
Perhaps their success is explained by the fact that they do many things differently than you would expect from a startup. The company Meshcapade is not based in Silicon Valley, London or Berlin, but in the German university town of Tübingen am Neckar. The CEO is not a man, but a woman: Naureen Mahmood, co-founder of the startup, 40 years old, headscarf, originally from Pakistan. And instead of keeping itself alive with capital from external investors, the young company has been in the black since the very beginning.
Naureen Mahmood.
Image: Meshcapade
Meshcapade has committed itself to a problem that technology companies are just as keen on as clothing companies, video game developers and health research: depicting people in virtual worlds as realistically as possible, with all their peculiarities in facial expressions and gestures.
So far, it has been surprisingly difficult to reproduce in digital space how each of us looks and moves, from the wrinkles on our foreheads to our wobbling stomachs. Instead, the avatar in Meta is missing its legs, in Apple’s case it is missing its entire body, and the faces usually look like they have been largely smoothed with Botox.
The startup Meshcapade, founded in 2018, has now built a computer model with which anyone can create their personal avatar in 3D in no time. All you have to do is upload a photo or video of yourself to a platform or answer a few questions about your height, weight and body shape.
The image created in this way can be given any movement pattern – an original hip-hop dance that you discovered on YouTube or a complex gymnastics choreography. Or you can dress your virtual self in outfits from online retailers. Doctors are also already using the startup’s technology to detect unusual movement patterns in small children, which could be the first indication of a cerebral movement disorder.
The simulation model called SMPL depicts movement patterns particularly realistically.
Image: Meshcapade
Their technology can capture every individual down to the folds of their clothes, explains Mahmood in an interview on the sidelines of a large video game trade fair in San Francisco. The gaming industry is also one of Meshcapade’s customers: developers of computer games can transfer human movement patterns to virtual characters. At the trade fair stand, Mahmood and her colleagues explain to interested visitors how their model works and how it should change the gaming industry.
Computers beat us at chess – but fail at “rock, paper, scissors”
“Our aim is to teach computers what people look like – and also how to interact with them,” says Mahmood, a woman perhaps 1 meter 60 tall with a hearty laugh. But computers are still surprisingly bad at interacting with people: they have been defeating people in chess for years, but the simplest interactive games like “rock, paper, scissors” cannot be played with the systems to this day.
Meshcapade is the result of years of research. At the Max Planck Institute for Intelligent Systems in Tübingen, Mahmood, her co-founders and other scientists worked for years on the so-called SMPL model (Skinned Multi-Person Linear Model), which creates a good and three-dimensional image of the human body should create.
The institute is one of the world‘s best research centers for this form of computer vision. This was also a reason why the computer specialist Mahmood moved to Tübingen after studying in the Pakistani city of Lahore and Texas. Together with the institute’s founder and computer scientist Michael Black and other colleagues, Mahmood trained the simulation model with more than a million 3D and 4D images of human bodies.
SMPL has now established itself as the standard for anyone who wants to depict people in three dimensions. They initially made the model available to other scientists free of charge, but inquiries from industry, such as clothing companies, increased. In 2018, Mahmood, the institute director Black and their colleague Talha Zaman founded Meshcapade as a spin-off of the Max Planck Institute.
Investor money not from Europe, but from Silicon Valley
From the very beginning, the startup made money by charging corporate customers a license fee for using the model. “We were able to grow on our own and concentrate entirely on building the platform,” says Mahmood.
After the platform went online last year, the company began looking for investor money in order to grow faster in the future. This did not happen in Germany or Europe, but in far away Silicon Valley.
In the American technology mecca, startup financing works much better than in Europe, says Mahmood. “Investors in Europe simply understand little about how we work and our industry.” California venture capital firm Matrix invested $6 million in Meshcapade.
Movement sequences like this from karate can be transferred from a video to a virtual character.
Image: Meshcapade
Alternatively, Mahmood and her co-founders could have sold their company to a tech company. They received takeover offers from Big Tech every month. But integrated into a corporate structure, they would not be able to work so freely on their own goals, believes the CEO: “We want a future in which we all have our own digital counterparts who interact with us and with each other.”
From your point of view, is the metaverse imminent? “In the next few years we will see enormous technological advances that will fundamentally change how we use technology and what it enables everyone.”
And in order to help shape this future and really drive progress, you have to be a startup and not be subject to the constraints of a corporation, believes Mahmood; Open AI has demonstrated this in the area of artificial intelligence.
If you move, it would be better to move to Switzerland than to Silicon Valley
Today, Meshcapade has 26 employees worldwide, from Switzerland to India to China. About a third of them are women. As a managing director, Mahmood herself always has to struggle with prejudices. At conferences or trade fairs like the one currently in San Francisco, it happens all the time that someone asks her about the company’s CEO. “Practically everyone is taken aback when I answer that it is me,” she says. But if you treat them with respect afterwards, the whole thing doesn’t bother them – in fact it’s now a running joke among the employees.
And why is Meshcapade still based in the small Neckar town of Tübingen – and not in a startup hub like Berlin or Silicon Valley? The proximity to the Max Planck Institute and the University of Tübingen is very important to them, both of which are among the top centers in their research area of computer vision. Zurich is also a hub, “if we move, it will be to Switzerland rather than to California.” However, Tübingen offers a tranquil, simple life, says Mahmood and laughs. “There aren’t a lot of distractions, and that’s good for our work.”