Reference-Driven Line Art Video Colorization

A raw line-art/sketch video sequence and a single colored reference image of the character are fed into a video diffusion model that uses point-tracking correspondences to transfer color and shading from the reference onto every sketch frame, producing a temporally consistent colored animation.

Properties

family: style-control
modality: line-art video + reference character image -> tracked correspondences -> colorized animation
whyMultiModel: A single video diffusion model cannot both (a) infer temporally-stable pixel correspondences between a static reference pose and a moving sketch character and (b) generate coherent color; a dedicated point-tracker supplies explicit correspondence guidance the diffusion U-Net conditions on, and an SVD-based backbone handles temporal coherence and generation separately.
steps: [object Object], [object Object], [object Object]
controls: reference character image (color/design sheet), point-tracking correspondence maps, per-frame sketch/line-art control, frame count (14-frame native window)
exampleStack: CoTracker2 -> AniDoc sketch extractor -> Stable Video Diffusion (AniDoc checkpoint)
useCases: [2D anime in-betweening automation][colorizing rough animatics from a character color sheet][reducing manual cel-coloring labor in production pipelines]
pitfalls: Large pose deviations between reference and sketch frames can cause color bleeding; native window is 14 frames so longer shots need chunking/stitching, which can introduce seams.
source: AniDoc: Animation Creation Made Easier (CVPR 2025, arXiv:2412.14173)

Provenance

Source: https://arxiv.org/abs/2412.14173

Properties

family

style-control

modality

line-art video + reference character image -> tracked correspondences -> colorized animation

whyMultiModel

A single video diffusion model cannot both (a) infer temporally-stable pixel correspondences between a static reference pose and a moving sketch character and (b) generate coherent color; a dedicated point-tracker supplies explicit correspondence guidance the diffusion U-Net conditions on, and an SVD-based backbone handles temporal coherence and generation separately.

steps

[object Object], [object Object], [object Object]

controls

reference character image (color/design sheet), point-tracking correspondence maps, per-frame sketch/line-art control, frame count (14-frame native window)

exampleStack

CoTracker2 -> AniDoc sketch extractor -> Stable Video Diffusion (AniDoc checkpoint)

useCases

[2D anime in-betweening automation][colorizing rough animatics from a character color sheet][reducing manual cel-coloring labor in production pipelines]

pitfalls

Large pose deviations between reference and sketch frames can cause color bleeding; native window is 14 frames so longer shots need chunking/stitching, which can introduce seams.

source

AniDoc: Animation Creation Made Easier (CVPR 2025, arXiv:2412.14173)