Geometry-Then-Texture 3D Asset Pipeline

A first model generates an untextured 3D geometry (mesh / TSDF) from a single image or text prompt; a second, separate diffusion model synthesises high-resolution texture maps conditioned on the produced geometry, yielding a fully textured, PBR-ready 3D asset.

Properties

family: 3d
modality: image -> 3D mesh -> textured 3D asset
whyMultiModel: Geometry generation and texture synthesis have conflicting objectives: the shape model optimises for accurate 3D form while the texture model optimises for view-consistent, photorealistic appearance conditioned on that form. Splitting into two specialist models lets each solve its own problem; the texture model explicitly conditions on the finished geometry rather than trying to hallucinate both simultaneously.
steps: [object Object], [object Object], [object Object]
controls: Input image quality and field of view; geometry resolution cap; texture diffusion guidance scale; number of synthesis views; UV unwrap quality; PBR channel selection (albedo, metallic, roughness, normal).
exampleStack: Step1X-3D geometry stage (VAE-DiT TSDF) -> Step1X-3D texture stage (SD-XL-fine-tuned multi-view generator) -> mesh export. Alternative: Hunyuan3D-DiT shape -> Hunyuan3D-Paint texture -> GLB.
useCases: [Game asset prototyping from concept art][Product visualisation from a single photo][3D printing prep from reference images][Rapid 3D scene population for VFX]
pitfalls: Texture seams and occlusion holes where multi-view coverage is sparse; UV inpainting quality degrades for highly concave surfaces; geometry errors propagate into texture conditioning (garbage-in, garbage-out); PBR decomposition may bake lighting into albedo.
source: Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets (arXiv:2505.07747)

Provenance

Source: https://arxiv.org/abs/2505.07747

Properties

family

modality

image -> 3D mesh -> textured 3D asset

whyMultiModel

Geometry generation and texture synthesis have conflicting objectives: the shape model optimises for accurate 3D form while the texture model optimises for view-consistent, photorealistic appearance conditioned on that form. Splitting into two specialist models lets each solve its own problem; the texture model explicitly conditions on the finished geometry rather than trying to hallucinate both simultaneously.

steps

[object Object], [object Object], [object Object]

controls

Input image quality and field of view; geometry resolution cap; texture diffusion guidance scale; number of synthesis views; UV unwrap quality; PBR channel selection (albedo, metallic, roughness, normal).

exampleStack

Step1X-3D geometry stage (VAE-DiT TSDF) -> Step1X-3D texture stage (SD-XL-fine-tuned multi-view generator) -> mesh export. Alternative: Hunyuan3D-DiT shape -> Hunyuan3D-Paint texture -> GLB.

useCases

[Game asset prototyping from concept art][Product visualisation from a single photo][3D printing prep from reference images][Rapid 3D scene population for VFX]

pitfalls

Texture seams and occlusion holes where multi-view coverage is sparse; UV inpainting quality degrades for highly concave surfaces; geometry errors propagate into texture conditioning (garbage-in, garbage-out); PBR decomposition may bake lighting into albedo.

source

Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assets (arXiv:2505.07747)