Generative AI for Stop-Motion Animation

<p dir="ltr">Stop-motion animation is traditionally created using the straight-ahead approach, where animators incrementally adjust physical objects and capture each frame sequentially. This is a time-consuming and repetitive process that limits the animator's ability to plan or edit complex scenes.</p><p dir="ltr">This research introduces a system that leverages recent advances in AI, including pose detection and pose transfer models, to generate stop-motion animations in a more flexible and efficient way. The proposed pipeline takes as input a video or sketches describing the motion of a character (pose reference frames) and photos of a figuring (character reference images). The appearance of the resulting animation would be fully controllable using the character reference images, and the motion would be fully controllable using the pose reference frames. The user can then make detailed edits to the animation using tools provided by the user interface.</p><p dir="ltr">Through qualitative evaluation on representative examples and quantitative evaluation on a dataset of animations, the system demonstrates that it can generate high-quality animations. The system can support animators through the entire stop-motion animation process, enabling use cases such as creating new animations through pose-to-pose or motion-capture techniques, or refining existing animations at the frame level. This would greatly reduce the workload, allowing more animators to explore stop-motion animation.</p>