A touch of life to computer animations

The chimpanzees, gorillas and orangutans that star in the hit film “Rise of the Planet of the Apes” are all computer animations. But they look a lot like the real thing, even to a primatologist.

“It’s astonishing how far the technology has come,” said Frans de Waal, a professor of primate behavior at Emory University in Atlanta and director of the Living Links Center at the Yerkes National Primate Research Center at Emory.

“We have the illusion we are looking at chimpanzees,” Dr. De Waal said of the computer-generated figures. “They are remarkably convincing.”
Producing computer-animated chimps that people will accept as realistic is a signal accomplishment, said Chris Bregler, an associate professor of computer science at New York University.

“It’s easier to fool us when you animate a dragon or another mythical or fairy tale creature,” Dr. Bregler said of characters created in earlier movies using the technology, called performance capture. “But humans or their closest relatives, chimps — that’s more difficult to do. Our human eyes are finely tuned to detecting problems with those depictions, and the illusion breaks down.”

If the illusion holds in “Rise of the Planet of the Apes,” he said, it is because of several small but significant engineering steps that can take audiences past what is called the “uncanny valley,” the spot at which an animation loses credibility.

In particular, “Rise” uses a video system that analyses the facial expressions of the actors playing the apes. “The system can capture every subtle nuance of expression down to the pixel,” Dr. Bregler said, “and every wrinkle. The wrinkles are especially important.”

Performance capture technology, as its name suggests, is based on actual performances by human actors. But In “Rise of the Planet of the Apes,” actors playing chimps do not wear heavy makeup and wigs, as Roddy McDowall did in the 1968 “Planet of the Apes.”

Instead, they wear special electronic gear. In “Rise,” the actor Andy Serkis, who plays the chimp protagonist Caesar, wears a lightweight helmet with a video camera mounted directly in front of his face to record every nuance of expression, outdoors as well as indoors.

These cameras were also employed in “Avatar,” but were used outside for the first time in “Rise,” said Mark Sagar, special projects supervisor at Weta Digital Ltd. and an Oscar winner for his facial motion capture techniques. Weta, a visual effects studio in New Zealand, provided the performance capture technology for “Rise” as well as for “Avatar” and earlier films including the “Lord of the Rings” trilogy.

The technology of performance capture was well on its way when Serkis played the mythical character Gollum in “Lord of the Rings,” but facial capture then was at lower resolution and more rudimentary. “The burden was much more on an animator to craft the facial expressions frame by frame,” Dr. Bregler said. “You couldn’t track every pixel in Gollum’s face.”

Moving the technology outside for filming has been another improvement in the evolution of motion capture." You need to do a lot of new engineering to make this work outside, Dr. Bregler said, but the effect is striking. “The actors are set in a natural environment. It’s much more realistic.”

All of the motion capture footage is analysed and modeled, becoming the basis for computer animation of the ape characters.

For example, when Caesar is called upon to frown — and Caesar does a lot of frowning as the plot thickens — the camera pointing at his face tracks the motion of his skin.

Software analyses how the underlying human muscles move, and then translates the movement to a chimp counterpart. “For example, an ape’s brow moves in a different way than an actor’s human brow,” Dr. Sagar said.

Software may do most of the animating, but human artists still apply their skills, adjusting the rendering if Serkis’s protruding human nose is not squashed exactly as it should be to become a chimp’s nose, or if the emotional intent of the performance is not conveyed properly.

“The process is not completely mathematical,” said Joe Letteri, a four-time Oscar winner and senior visual effects supervisor for Weta. Weta worked closely with Standard Deviation, a motion capture hardware company in Santa Monica, Calif., to design and build many of the cameras used in “Rise,” said Dejan Momcilovic, head of performance capture at Weta. Cameras from the Motion Analysis Corporation in Santa Rosa, Calif., were also used, he said. The live action shots, combined with the convincing computerized faces of the chimps, work to increase realism, said Steven R. Quartz, a professor of philosophy at the California Institute of Technology in Pasadena. “Caesar really seems to be interacting with the other actors. It really does seem to capture subtle nuances in the interaction and the exchanges,” Dr. Quartz said, even though Caesar is artificial.

However plausible the animation, movie audiences are not actually seeing a true representation of a chimp’s brow, or eyes either, when they look at Caesar, Dr. De Waal of Emory said. There is plenty of digital fine-tuning to humanise the apes, making them a bit more like us and a bit less like them.

“The head and shoulders of an actual chimp are more massive than Caesar’s,” he said. And Caesar’s eyes have been altered, too, for example, to add white to them.
But Dr. De Waal is glad the filmmakers used actors rather than chimps that are dressed up to play parts. The animals “have miserable lives dancing to our tune,” he said. “We are very pleased there is a movie that has been able to circumvent this.”