DoodleAssist: Progressive Interactive Line Art Generation
with Latent Distribution Alignment
Haoran Mo1,  Yulin Shen1,  Edgar Simo-Serra2,  Zeyu Wang1,3
1The Hong Kong University of Science and Technology (Guangzhou),  2Waseda University, 
3The Hong Kong University of Science and Technology

Accepted by IEEE Transactions on Visualization and Computer Graphics
(TVCG 2025)
[Paper]  [Paper (IEEE)]  [Code]


Workflow of our interactive and progressive line art generation system DoodleAssist, and its performance on diverse generation. (a) The progressive process generates or updates designated regions (in pink) step by step according to drawn or modified strokes in the sketches, while ensuring smooth transition with those unchanged regions. The actions at each step are shown with color coding. (b) The system can be applied to synthesize diverse line art images, such as single characters, animals, objects, multiple characters, and complex scenes.
Abstract
Creating high-quality line art in a fast and controlled manner plays a crucial role in anime production and concept design. We present DoodleAssist, an interactive and progressive line art generation system controlled by sketches and prompts, which helps both experts and novices concretize their design intentions or explore possibilities. Built upon a controllable diffusion model, our system performs progressive generation based on the last generated line art, synthesizing regions corresponding to drawn or modified strokes while keeping the remaining ones unchanged. To facilitate this process, we propose a latent distribution alignment mechanism to enhance the transition between the two regions and allow seamless blending, thereby alleviating issues of region incoherence and line discontinuity. Finally, we also build a user interface that allows the convenient creation of line art through interactive sketching and prompts. Qualitative and quantitative comparisons against existing approaches and an in-depth user study demonstrate the effectiveness and usability of our system. Our system can benefit various applications such as anime concept design, drawing assistant, and creativity support for children.
[Paper]    [Paper (IEEE)]    [Supplementary]    [Code]   
Method

Framework Overview



Framework of our progressive line art generation system DoodleAssist. (a) It is built by applying a controllable diffusion model iteratively to update intended regions for incoming or modified strokes. (b) Regional latent blending enables the progressive process with region control, which accounts for noise prediction and denoising in the intended region M. The remaining region 1−M is replaced with a known latent from the last generated line art. (c) A latent distribution alignment mechanism is proposed to improve transitions between the two regions. Newly added or modified strokes are highlighted in red. Masks in pink indicate the intended regions.

Overall Introduction

(Or watch on Bilibili)
👇

Results

Result Gallery



Support for different creation workflows of participants in the user study. We plot their different actions along a timeline using color coding. The four results of each participant correspond to the orders on the timeline.

Comparisons 1



Comparisons with line art generation methods based on complete or iterative sketches. Newly added strokes are highlighted in red.

Comparisons 2



User study of iterative sketch to line art generation between Block-and-Detail system and ours. Newly added or modified strokes are highlighted in red. Blue drawings underneath are the last generated images.

BibTeX
@article{mo2025doodleassist,
  title={DoodleAssist: Progressive Interactive Line Art Generation with Latent Distribution Alignment},
  author={Mo, Haoran and Shen, Yulin and Simo-Serra, Edgar and Wang, Zeyu},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  year={2025},
  publisher={IEEE}
}