Animated Features Rely on Artists as AI Works in Support

As technological advances make even more astonishing imagery possible for many of the leading contenders in the animated feature Oscar race, they’re still leaning toward the skilled hands of the artisans who’ve mastered their craft and away from AI or machine learning.

While some may use machine learning — a type of AI — in the early stages of motion capture or to iterate ideas, the knowledge of experienced VFX artists and the performance capabilities of animators are central to creating the unique imagery sought by the filmmakers behind these movies. Some films, like “Lord of the Rings: The Lord of the Rohirrim,” are more focused on traditional low-tech tools to create their look.

“Transformers One,” an origin story helmed by Josh Cooley, was designed with heavy research on both previous Transformers storytelling and the types of metals that could be used to create the impact the filmmakers wanted. Cooley’s team consisted of many artists who grew up on Transformers and had strong opinions of how they should look and move. VFX supervisor Frazer Churchill points out that while they used some machine learning early in the process of making the film in conjunction with motion capture techniques, none of it made it into the final film.

“This film is really just made by artistry and time,” says Churchill. “Our first touchstone was that we need the metal to feel real. The audience needs to feel that they can reach out and touch the metallic characters. We worked at building up a huge library of metal surfaces for our starting point, working with the surfacing artists, looking at various types of metal and painted metals. Then we worked on using very saturated colors, which kind of take [the film] to a more stylized feel. All of our lighting is what we call plausible lighting. We have lots of different lights placed around the place since we’re going for a cinematic backlit look with characters that are backlit, like Megatron, with a beautiful rim light on him. Artists are making these decisions.”

Cooley is quick to point out that at this stage, “AI can only spit out what you put into it,” so for innovative ideas, he needs a crew of experienced filmmakers who are excited about working on a Transformers project.

Ironically, in “The Wild Robot,” a story about a robot who comes to develop a close relationship with a gosling that it raises, AI was not a tool that made sense for the filmmakers. VFX supervisor Jeff Budsberg took a painterly approach to the VFX in the film since helmer Chris Sanders wanted the feature to look more like a fairy tale, with a less detailed and more impressionistic feeling overall. It’s this vision that kept the film in line with the world described in the book and also created a warmer feeling throughout
the movie.

“There’s something strategic about where you place detail and where you remove superfluous information, guiding the audience’s eye with a lot of intent,” says Budsberg. “Chris [Sanders], very early on, said he wanted to pay homage to those early Disney films like ‘Bambi.’ He was also influenced by Miyazaki films. Those environments are just gorgeous. There’s something about those films that just is endearing and heartfelt and captivating. He challenged me and the team to bring that to life. I was coming off of ‘The Bad Guys,’ which was one of our first explorations into this kind of non-photo realistic style. [‘The Bad Guys’] was more of a kind of homage to comic books or illustration, kind of like drawing with a Prismacolor marker instead. That step was the first step that allowed us to do [2022’s] ‘Puss in Boots.’ And ‘Puss in Boots’ was a dramatically different style, more fantasy. It’s very interesting to be doing this hybrid 2D and 3D style. Right now we don’t use AI in Dreamworks films at all. It’s still a very Wild West technology from our point of view. And it’s not specific enough for what we want to do with the look of ‘The Wild Robot.’”

At Pixar, AI has been part of certain aspects of animated filmmaking for many years. The main craftsmanship and innovation is still led by artists but other tasks — like denoising certain imagery — are assisted with AI or machine learning. In the case of “Inside Out 2,” RenderMan 25 was used to remove visual noise with the help of machine learning technology. This tech isn’t push-button; it still requires artists to monitor what it does to make sure it has the desired impact.

For “Piece by Piece,” the animated Lego biopic about musician/producer/entrepreneur Pharrell Williams, helmer Morgan Neville searched for ways to make the film’s Lego characters more approachable and human, especially since many of them are famous musicians who Williams meets along the way during this journey. Neville decided basing the Lego characters on the people interviewed for the biopic was best, since many of them are also huge figures in music. The filmmaker and his crew used scanning tech to create and map the faces of people featured in the film who were then turned into Lego characters.

“There were specific rules about how we could have the Lego characters move and what they could do because Lego has rules for that,” says Neville. “It’s all based on what Legos can do in real life, so you have to live within those rules and that universe. There was still a lot we could do and things that we had to figure out like making Legos look like water that was moving and having a boat float off on some water — all made of Lego in animation — but we had great artists who were able to work with that challenge.”

“Lord of the Rings: The Lord of the Rohirrim,” with its roots in traditional Japanese 2D animation and use of a hand-drawn aesthetic, embraces a more painterly approach in everything from the characters to the lighting. Though AI wasn’t part of the process on this “LOTR” film, CGI animation supervisor Kei Yoshimizu sees it as part of the future of Japanese animation.

“There’s no doubt that AI will gradually become more integrated into various aspects of the anime production industry in Japan,” says Yoshimizu through a translator. “This is similar to how the introduction of 3D CGI was in the 1990s. For this project, for example, 3D CGI serves a more ‘behind-the-scenes’ role, while other works may showcase more visible 3D elements. Regardless, these technologies are all employed with the goal of creating anime, and full 3D productions that forgo hand-drawn techniques are still rare in Japan. In essence, 3D CGI has become an essential part of the anime production process without completely replacing hand-drawn animation,” he says.

On the flip side, Yoshimizu adds, “hand-drawn animation faces a significant challenge with the shortage of skilled personnel to meet the immense production demands, making the efficiency provided by AI a crucial factor to consider. I personally feel that, similar to the case with 3D CGI animation, the industry will seek ways for AI and traditional techniques to coexist. Ultimately, how well AI is accepted will likely be dictated by the market. Currently, it seems that most viewers of Japanese anime still deeply value the hand-drawn ‘key frame’ style of animation.”