GLM 4.7

[reasoning]

Z.AI[text][open-weights]

Enhanced programming and stable multi-step reasoning with agent task execution

Interactive Architecture

Take GLM 4.7 apart, layer by layer. The 3D teardown below shows its 92 transformer layers (3 dense + 89 MoE), GQA attention, and expert routing, with values from the public model config.

92 layers (3 dense → 89 MoE)

GQA attention · 96Q heads

8+1 of 160 experts active

355B params · 32B active

Token embedding

5120-dim vectors over a 151,552-token vocabulary. GLM-4.7 is the agentic-coding flagship of the GLM-4 line.

92 layers (3 dense prefix)

A deep pre-norm stack: the first 3 layers use a dense FFN, the remaining 89 are MoE. QK-Norm stabilises attention across the 92-layer depth, and a 1-layer Multi-Token-Prediction head speeds decoding.

Grouped-query attention

96 query heads share 8 KV heads via GQA (head_dim 128) with QK-Norm and 50% partial rotary embeddings.

160-expert MoE

A router scores 160 experts per token and activates 8 plus 1 shared (expert FFN 1536), spending 32B of 355B per token.

Output head

Final RMSNorm then projection to 151,552 logits. GLM-4.7 adds Interleaved and Preserved Thinking for stable multi-turn agentic coding.

Input

$0.40

per 1M tokens

Output

$1.50

per 1M tokens

Context

203K tokens

Max Output

66K tokens

Released

2025-12-22

Features

[optional-reasoning][programming][multi-step-reasoning][agent-tasks][tool-calling]

Technical Specifications

LicenseMIT

Architecture

ArchitectureTransformer

ParametersUnknown (proprietary)

Resources

HuggingFace Demo

Benchmarks

AIME 202595.7%

GPQA Diamond85.7%

LiveCodeBench84.9%

MMLU-Pro84.3%

SWE-bench Multilingual66.7%

SWE-bench Verified73.8%

Tau2-bench87.4%

Terminal-Bench41.0%

The Lab

location: Beijing, Chinafounded: 2019backed by: Listed on HKEX (2513); early investors include Alibaba, Tencent, HongShan

Founded in 2019 as a Tsinghua University spinout, the company was known internationally as Zhipu AI until it rebranded as Z.ai in July 2025; it is headquartered in Beijing and has been listed on the Hong Kong Stock Exchange since January 2026. The lab develops the GLM (General Language Model) family, released under the MIT licence since GLM-4.5 in July 2025 and focused on coding and agentic tasks, alongside the GLM-V vision line and the AutoGLM phone agent.

more about Z.ai (Zhipu AI) →

How It Works

What vocabulary and embedding size does GLM 4.7 use?+

5120-dim vectors over a 151,552-token vocabulary. GLM-4.7 is the agentic-coding flagship of the GLM-4 line.

How many transformer layers does GLM 4.7 have?+

A deep pre-norm stack: the first 3 layers use a dense FFN, the remaining 89 are MoE. QK-Norm stabilises attention across the 92-layer depth, and a 1-layer Multi-Token-Prediction head speeds decoding.

What attention does GLM 4.7 use?+

96 query heads share 8 KV heads via GQA (head_dim 128) with QK-Norm and 50% partial rotary embeddings.

How does the Mixture of Experts in GLM 4.7 work?+

A router scores 160 experts per token and activates 8 plus 1 shared (expert FFN 1536), spending 32B of 355B per token.

How does GLM 4.7 produce output tokens?+

Final RMSNorm then projection to 151,552 logits. GLM-4.7 adds Interleaved and Preserved Thinking for stable multi-turn agentic coding.

scene values sourced from:HuggingFace config HuggingFace config

Model ID: glm-4.7

Interactive Architecture

Take GLM 4.7 apart, layer by layer. The 3D teardown below shows its 92 transformer layers (3 dense + 89 MoE), GQA attention, and expert routing, with values from the public model config.

92 layers (3 dense → 89 MoE)

GQA attention · 96Q heads

8+1 of 160 experts active

355B params · 32B active

Token embedding

5120-dim vectors over a 151,552-token vocabulary. GLM-4.7 is the agentic-coding flagship of the GLM-4 line.

92 layers (3 dense prefix)

A deep pre-norm stack: the first 3 layers use a dense FFN, the remaining 89 are MoE. QK-Norm stabilises attention across the 92-layer depth, and a 1-layer Multi-Token-Prediction head speeds decoding.

Grouped-query attention

96 query heads share 8 KV heads via GQA (head_dim 128) with QK-Norm and 50% partial rotary embeddings.

160-expert MoE

A router scores 160 experts per token and activates 8 plus 1 shared (expert FFN 1536), spending 32B of 355B per token.

Output head

Final RMSNorm then projection to 151,552 logits. GLM-4.7 adds Interleaved and Preserved Thinking for stable multi-turn agentic coding.

The Lab

location: Beijing, Chinafounded: 2019backed by: Listed on HKEX (2513); early investors include Alibaba, Tencent, HongShan

more about Z.ai (Zhipu AI) →

How It Works

What vocabulary and embedding size does GLM 4.7 use?+

5120-dim vectors over a 151,552-token vocabulary. GLM-4.7 is the agentic-coding flagship of the GLM-4 line.

How many transformer layers does GLM 4.7 have?+

A deep pre-norm stack: the first 3 layers use a dense FFN, the remaining 89 are MoE. QK-Norm stabilises attention across the 92-layer depth, and a 1-layer Multi-Token-Prediction head speeds decoding.

What attention does GLM 4.7 use?+

96 query heads share 8 KV heads via GQA (head_dim 128) with QK-Norm and 50% partial rotary embeddings.

How does the Mixture of Experts in GLM 4.7 work?+

A router scores 160 experts per token and activates 8 plus 1 shared (expert FFN 1536), spending 32B of 355B per token.

How does GLM 4.7 produce output tokens?+

Final RMSNorm then projection to 151,552 logits. GLM-4.7 adds Interleaved and Preserved Thinking for stable multi-turn agentic coding.