Chung-Yi Weng, PhD student at the University of Washington, and some of his friends created something truly astonishing.
Their software called “Photo Wake-Up” allows character animations to simply “walk out” of a static image frame — without leaving a hole in the picture behind them. The results were published in a recently submitted paper.
Weng’s method identifies a 2D subject in a single photo as input, and creates a 3D animated version of that subject. The animation can then “walk out, run, sit, or jump in 3D.”
Researchers have tried to create a similar effect in the past, but the results have been a lot less impressive. Weng’s new approach adds an important new ability: it can identify different body parts like arms and legs, and warp each one individually in a way that matches the 2D cutout exactly.