The eerie, unsettling atmosphere of “The Frost” is evident from the very first shots. One sees icy mountains, a makeshift military camp, people huddled around a fire, dogs barking, creating a growing sense of dread: Something is wrong here.
“Give me the cock,” says a voice. A close-up shows a man by the fire gnawing on a piece of jerky. The way his lips move isn’t quite right. For a moment, it looks like he’s chewing on his own frozen tongue.
Welcome to the grotesque world of AI movies. “At some point we stopped insisting on photographic accuracy and started embracing the weirdness of DALL-E,” says Stephen Parker of Waymark, the Detroit-based video production company behind The Frost. The result is a 12-minute film (exclusively viewable on the American MIT Technology Review website) in which every shot is generated by an AI. It’s one of the most impressive – and bizarre – examples of this strange new genre to date.
For the production of “The Frost,” Waymark took a script from Josh Rubin, an executive producer for the company, and fed it to the DALL-E 2 AI image generator. After a few attempts to get the model to produce images in a style , which they were satisfied with, the filmmakers used DALL-E to “shoot” each additional shot. Then they used D-ID, a tool that can be used to animate still images. This allows the tents to flap in the wind and lips to move when speaking.
AI movies on the rise
“This is certainly the first generative AI film I’ve seen where the style feels consistent,” says Souki Mehdaoui, an independent filmmaker and co-founder of Bell & Whistle, a creative technology consultancy. “The generation of still images and the later puppetry gives the film a special collage vibe.”
“The Frost” is not the only short film that has been created using generative artificial intelligence in recent months. Even the best models are currently limited to just a few seconds of video. As a result, many films exhibit a wide range of styles and techniques, ranging from storyboard-like sequences, as in “The Frost,” to wild juxtapositions of many different video clips.
In February and March, Runway, a US company that makes AI tools for video production, held an AI film festival in New York. Highlights included Laen Sanches’ “PLSTC,” a dizzying sequence of strange plastic-wrapped sea creatures generated using the Midjourney image generator. “Given Again” by Jake Oleson, which uses a technology called NeRF (neural radiance fields) that turns 2D photos into virtual 3D objects. And the surreal nostalgia of Sam Lawton’s “Expanded Childhood,” a slideshow of old family photos that Lawton used DALL-E 2 to expand beyond its limits, allowing him to play with the half-remembered details of the old images.
Recommended Editorial Content
With your consent, an external YouTube video (Google Ireland Limited) will be loaded here.
Always load YouTube video
Expanded Childhood — Sam Lawton
Fast and cheap
Artists are often the first to experiment with new technologies. But the advertising industry is also often at the forefront. Waymark also developed “The Frost” to find out how the company could integrate generative AI into its products. Waymark makes video production software that companies can use to create commercials quickly and cheaply.
Waymark’s current technology, launched earlier this year, combines various AI techniques, including language models, image recognition and speech synthesis, to create an ad in just a few steps. Waymark also draws on its large dataset of traditional commercials created for previous clients. “We have hundreds of thousands of videos,” says CEO Alex Persky-Stern. “We picked the best ones and trained the AI on them to show it what a good video looks like.”
To use the Waymark tool, which is offered as part of a tiered subscription starting at $25 per month, users identify their company’s website or social media accounts. The software then collects all the text and images it can find. A commercial is then created from this data, with GPT-3 writing an appropriate script that is read aloud by a synthetic voice over selected images that highlight the company. A one-minute commercial is created in a matter of seconds. Clients can edit the result as they wish by tweaking the script, editing the images or choosing a different voice. Waymark says more than 100,000 people have used the tool so far.