Loading...
Loading...
Flagship open-source model for complex systems and agent workflows
Take GLM 5 apart, layer by layer. The 3D teardown below shows its 78 transformer layers (3 dense + 75 MoE), MLA attention, and expert routing, with values from the public model config.
Founded in 2019 as a Tsinghua University spinout, the company was known internationally as Zhipu AI until it rebranded as Z.ai in July 2025; it is headquartered in Beijing and has been listed on the Hong Kong Stock Exchange since January 2026. The lab develops the GLM (General Language Model) family, released under the MIT licence since GLM-4.5 in July 2025 and focused on coding and agentic tasks, alongside the GLM-V vision line and the AutoGLM phone agent.
more about Z.ai (Zhipu AI) →6144-dim vectors over a 154,880-token vocabulary, up from 151,552 in GLM-4.5. The base model was pre-trained on 28.5 trillion tokens.
A pre-norm stack of 78 decoder layers: the first 3 use a dense FFN, the remaining 75 are MoE. GLM-5 is shallower than GLM-4.7 (92 layers) to cut expert-parallel communication. One Multi-Token Prediction layer sits on top, with its parameters shared across 3 prediction steps in training to lengthen speculative-decoding accept length.
Multi-head Latent Attention with 64 heads: queries pass through a 2048-dim LoRA, keys and values through a 512-dim latent. Z.AI cut the head count by a third and widened each head to 256 dims to lower decoding cost. A DSA indexer (32 heads, 128 dims) then keeps only the top 2048 cached tokens for each query.
A sigmoid router picks 8 of 256 routed experts per token, plus 1 shared expert (expert FFN 2048). 40B of 744B parameters fire per token, against 32B of 355B in GLM-4.5.
Final RMSNorm then an untied projection to 154,880 logits. Sparse attention keeps the 202,752-token context affordable at 744B scale.