Joint Stroke Tracing and Correspondence for 2D Animation
Haoran Mo,  Chengying Gao*,  Ruomei Wang
Sun Yat-sen University

Accepted by ACM Transactions on Graphics
(Presented at SIGGRAPH 2024)
[Paper]  [Code]
Given consecutive raster keyframes and a single vector drawing from the starting keyframe only, our method generates vector images for the remaining keyframes with one-to-one stroke correspondence. The framework trained with clean line drawings generalizes well to rough sketches. The generated results can be directly imported into an inbetweening system CACANi to produce inbetween frames to form 2D animation. Gunman from paper[1] is courtesy of Eugene Babich ©2018 John Wiley & Sons Ltd. Bigvegas2 is from Creative Flow+ dataset[2].
Abstract
To alleviate human labor in redrawing keyframes with ordered vector strokes for automatic inbetweening, we for the first time propose a joint stroke tracing and correspondence approach. Given consecutive raster keyframes along with a single vector image of the starting frame as a guidance, the approach generates vector drawings for the remaining keyframes while ensuring one-to-one stroke correspondence. Our framework trained on clean line drawings generalizes to rough sketches and the generated results can be imported into inbetweening systems to produce inbetween sequences. Hence, the method is compatible with standard 2D animation workflow. An adaptive spatial transformation module (ASTM) is introduced to handle non-rigid motions and stroke distortion. We collect a dataset for training, with 10k+ pairs of raster frames and their vector drawings with stroke correspondence. Comprehensive validations on real clean and rough animated frames manifest the effectiveness of our method and superiority to existing methods.
[Paper]    [Supplementary]    [Code]    [Vector results]   
Method

Framework Overview



Our framework takes as inputs consecutive raster frames, denoted as reference and target, along with a vector drawing of the reference containing several stroke chains (i.e., long curves), each of which comprises connected strokes. It performs a joint stroke tracing and correspondence task by generating corresponding vector strokes one by one (a). It consists of two models: one for matching starting point of each stroke chain (b), the other for predicting parameters of the associated strokes (c). The whole process works in a local view based on patches cropped by windows. A proposed plug-and-play adaptive spatial transformation module (ASTM) is integrated into the two models to handle large motions or stroke distortion.

Overall Introduction

(Or watch on Bilibili)
👇

Results
Our framework is applicable to real clean and rough animated frames of various resolutions and with fairly complex motions.

Clean Line Drawings

Hand from paper[3] ©2010 Blackwell Publishing Ltd. Eagle from paper[4] is courtesy of Jie Li ©2018, IEEE.
Stick from paper[4] is courtesy of Jie Li ©2018, IEEE.
Car from paper[4] is courtesy of Jie Li ©2018, IEEE.

Rough Sketches

These examples are from Creative Flow+ dataset[2]. Please see their original rough frames in the following video.

More results of rough sketches

(Or watch on Bilibili)
👇


Comparisons of inbetweening results

(Or watch on Bilibili)
👇


[Download Vector Results (354KB)]
BibTeX
@article{mo2024joint,
  title={Joint Stroke Tracing and Correspondence for 2D Animation},
  author={Mo, Haoran and Gao, Chengying and Wang, Ruomei},
  journal={ACM Transactions on Graphics},
  year={2024},
  publisher={ACM New York, NY}
}

References
[1] Wenwu Yang, Hock-Soon Seah, Quan Chen, Hong-Ze Liew, and Daniel Sýkora. FTP-SC: Fuzzy Topology Preserving Stroke Correspondence. SCA 2018. [Paper] [Webpage]
[2] Maria Shugrina, Ziheng Liang, Amlan Kar, Jiaman Li, Angad Singh, Karan Singh, and Sanja Fidler. Creative flow+ dataset. CVPR 2019. [Paper] [Webpage]
[3] Brian Whited, Gioacchino Noris, Maryann Simmons, Robert W Sumner, Markus Gross, and Jarek Rossignac. BetweenIT: An Interactive Tool for Tight Inbetweening. CGF 2010. [Paper]
[4] Wenwu Yang. Context-Aware Computer Aided Inbetweening. TVCG 2017. [Paper]