Loading...
Loading...
Free 12B multimodal reasoning model with hybrid architecture for OCR, chart reasoning, and vision tasks
[interactive architecture ↓]The AI research division of NVIDIA, the Santa Clara-based computing giant founded in 1993. NVIDIA AI Research develops foundation models, simulation platforms, and synthetic data generation tools that run on NVIDIA hardware, spanning LLMs, robotics world models, and drug discovery. The Nemotron family targets enterprise language tasks, while Cosmos builds generative world models for robotics and autonomous systems.
more about NVIDIA AI Research →Take Nemotron Nano 12B v2 VL apart, layer by layer. The 3D teardown below shows its 62 transformer layers and GQA attention, with values from the public model config.
5120-dim vectors over a 131,072-token vocabulary. A vision encoder feeds image tokens into this same stream for the VL variant.
A deep, fully-dense transformer (no MoE): every token passes through every layer's full FFN. Depth over width is the Nano design choice for a small efficient model.
40 query heads share 8 KV heads via GQA, balancing quality and KV-cache size for on-device and edge deployment.
A single dense FFN with a 20,480 intermediate dim and Squared ReLU activation. No experts: all 12.6B params are active for every token.
Final RMSNorm then 131,072 logits. The VL variant routes image and text tokens through this same head for unified multimodal output.