Hunyuan Hy3

Open-weight 295B (21B active) MoE with hybrid fast-and-slow thinking, strong on software development and productivity tasks

Interactive Architecture

Take Hunyuan Hy3 apart, layer by layer. The 3D teardown below shows its 80 transformer layers (1 dense + 79 MoE), GQA attention, and expert routing, with values from the public model config.

80 layers (1 dense → 79 MoE)

GQA attention · 64Q heads

8+1 of 192 experts active

295B params · 21B active

Token embedding

4096-dim vectors over a 120,832-token vocabulary.

80 layers (1 dense prefix)

A deep pre-norm stack: only layer 0 uses a dense FFN, the remaining 79 are MoE. QK-Norm stabilises the attention scores across the deep stack.

Grouped-query attention

64 query heads share 8 KV heads via GQA, with QK-Norm on the query/key projections for training stability.

192-expert MoE

A router scores 192 experts and activates 8 plus 1 shared (intermediate 1536), spending 21B of 295B per token. The relatively small expert FFN keeps each expert cheap.

Output head

Final RMSNorm then projection to 120,832 logits. Hy3 targets Tencent's product stack at 256K context.

Input

$0.20

per 1M tokens

Output

$0.80

per 1M tokens

Context

262K tokens

Max Output

131K tokens

Released

2026-07-06

Features

[open-weights-(apache-2.0)][hybrid-fast/slow-reasoning][256k-context][agentic-coding][cost-efficient]

Technical Specifications

LicenseApache 2.0

Benchmarks

BrowseComp84.2%

GPQA Diamond90.4%

Humanity's Last Exam37.0%

MCP Atlas79.1%

SWE-bench Multilingual75.8%

SWE-bench Pro57.9%

SWE-bench Verified78.0%

Terminal-Bench71.7%

The Lab

location: Shenzhen, Chinafounded: 2023backed by: Tencent Holdings

Hunyuan is the foundation-model brand of Tencent Holdings, the Shenzhen company founded in 1998; the Hunyuan large language model was launched on 7 September 2023 at Tencent's Global Digital Ecosystem Summit and is served through Tencent Cloud and products such as WeChat. Most recent releases are open weights: Hunyuan-Large (November 2024), HunyuanVideo (December 2024), the Hunyuan3D 2.x series (2025), HunyuanImage 2.1 and 3.0 (September 2025) and the 295B Hy3 language model (July 2026, Apache 2.0). Hunyuan 3D 3.0, announced in September 2025, is served through Tencent's Hunyuan 3D platform and API.

more about Tencent Hunyuan →

How It Works

What vocabulary and embedding size does Hunyuan Hy3 use?+

4096-dim vectors over a 120,832-token vocabulary.

How many transformer layers does Hunyuan Hy3 have?+

A deep pre-norm stack: only layer 0 uses a dense FFN, the remaining 79 are MoE. QK-Norm stabilises the attention scores across the deep stack.

What attention does Hunyuan Hy3 use?+

64 query heads share 8 KV heads via GQA, with QK-Norm on the query/key projections for training stability.

How does the Mixture of Experts in Hunyuan Hy3 work?+

A router scores 192 experts and activates 8 plus 1 shared (intermediate 1536), spending 21B of 295B per token. The relatively small expert FFN keeps each expert cheap.

How does Hunyuan Hy3 produce output tokens?+

Final RMSNorm then projection to 120,832 logits. Hy3 targets Tencent's product stack at 256K context.

scene values sourced from:HuggingFace config HuggingFace config

Model ID: hunyuan-hy3

Interactive Architecture

Take Hunyuan Hy3 apart, layer by layer. The 3D teardown below shows its 80 transformer layers (1 dense + 79 MoE), GQA attention, and expert routing, with values from the public model config.

80 layers (1 dense → 79 MoE)

GQA attention · 64Q heads

8+1 of 192 experts active

295B params · 21B active

Token embedding

4096-dim vectors over a 120,832-token vocabulary.

80 layers (1 dense prefix)

A deep pre-norm stack: only layer 0 uses a dense FFN, the remaining 79 are MoE. QK-Norm stabilises the attention scores across the deep stack.

Grouped-query attention

64 query heads share 8 KV heads via GQA, with QK-Norm on the query/key projections for training stability.

192-expert MoE

A router scores 192 experts and activates 8 plus 1 shared (intermediate 1536), spending 21B of 295B per token. The relatively small expert FFN keeps each expert cheap.

Output head

Final RMSNorm then projection to 120,832 logits. Hy3 targets Tencent's product stack at 256K context.

The Lab

location: Shenzhen, Chinafounded: 2023backed by: Tencent Holdings

more about Tencent Hunyuan →

How It Works

What vocabulary and embedding size does Hunyuan Hy3 use?+

4096-dim vectors over a 120,832-token vocabulary.

How many transformer layers does Hunyuan Hy3 have?+

A deep pre-norm stack: only layer 0 uses a dense FFN, the remaining 79 are MoE. QK-Norm stabilises the attention scores across the deep stack.

What attention does Hunyuan Hy3 use?+

64 query heads share 8 KV heads via GQA, with QK-Norm on the query/key projections for training stability.

How does the Mixture of Experts in Hunyuan Hy3 work?+

A router scores 192 experts and activates 8 plus 1 shared (intermediate 1536), spending 21B of 295B per token. The relatively small expert FFN keeps each expert cheap.

How does Hunyuan Hy3 produce output tokens?+

Final RMSNorm then projection to 120,832 logits. Hy3 targets Tencent's product stack at 256K context.