LLM-Orchestrated Talking-Scene Assembly

A general-purpose LLM first drafts the dialogue, per-shot prompts, and voice casting from a character brief; an image model then generates and edits successive scene keyframes, a video model animates each keyframe into a shot stitched via last-frame continuation, a talking-head model adds lip-synced performance and face consistency, and a voice model supplies the matching speech track.

Properties

family: audio-driven
modality: character brief -> keyframes -> video + talking-head -> voiced final cut
whyMultiModel: No single video model renders and edits a sequence of scene keyframes, animates those into shots, drives lip-synced facial performance, and synthesizes a timed voice track; each is a distinct trained model family chained by an LLM-authored shot plan that keeps them in sync via shared scene/character state.
steps: [object Object], [object Object], [object Object] +2 more
controls: character brief/persona, per-shot LLM-authored prompts, LoRA selection, last-frame continuation between segments, voice profile, audio-duration-driven video timing
exampleStack: Qwen-Image -> Qwen-Image-Edit -> Wan 2.2 I2V -> InfiniteTalk -> MiniMax voice synthesis
useCases: [AI-hosted talking-scene shorts with dialogue][Synthetic interview/skit production][Multi-scene narrative shorts with lip-synced characters]
pitfalls: Because timing is driven by measured audio duration feeding back into video segment length, a mismatch anywhere in the LLM-authored shot plan propagates through every downstream stage and only surfaces as broken sync in the final assembly.
source: lovisdotio/ComfyUI-Workflow-Sora2Alike-Full-loop-video

Provenance

Source: https://github.com/lovisdotio/ComfyUI-Workflow-Sora2Alike-Full-loop-video

LLM-Orchestrated Talking-Scene Assembly

Properties

family

audio-driven

modality

character brief -> keyframes -> video + talking-head -> voiced final cut

whyMultiModel

No single video model renders and edits a sequence of scene keyframes, animates those into shots, drives lip-synced facial performance, and synthesizes a timed voice track; each is a distinct trained model family chained by an LLM-authored shot plan that keeps them in sync via shared scene/character state.

steps

[object Object], [object Object], [object Object] +2 more

controls

character brief/persona, per-shot LLM-authored prompts, LoRA selection, last-frame continuation between segments, voice profile, audio-duration-driven video timing

exampleStack

Qwen-Image -> Qwen-Image-Edit -> Wan 2.2 I2V -> InfiniteTalk -> MiniMax voice synthesis

useCases

[AI-hosted talking-scene shorts with dialogue][Synthetic interview/skit production][Multi-scene narrative shorts with lip-synced characters]

pitfalls

Because timing is driven by measured audio duration feeding back into video segment length, a mismatch anywhere in the LLM-authored shot plan propagates through every downstream stage and only surfaces as broken sync in the final assembly.

source

lovisdotio/ComfyUI-Workflow-Sora2Alike-Full-loop-video