Qwen3.8 27B

[reasoning]

Qwen[text][open-weights]

Open-weight dense vision-language model for coding, research and long-running agent tasks, with thinking you can switch on or off

Interactive Architecture

Take Qwen3.8 27B apart, layer by layer. The 3D teardown below shows its 64 layers and GQA attention, with values from the public model config.

64 layers

GQA attention · 24Q heads

27B params

Token embedding

5120-dim vectors over a 248,320-token vocabulary. Qwen3.8 27B is natively vision-language: image and video tokens join this stream (the encoder is not drawn).

64 layers, 3:1 hybrid

Three Gated DeltaNet layers for every Gated Attention layer (48 + 16). DeltaNet is linear attention with a fixed-size state, so only a quarter of the stack grows a cache with context.

Gated DeltaNet + Gated Attention

Every fourth layer is full attention: 24 query heads share 4 KV heads of 256 dims, and an output gate scales what each head contributes. Rotary positions cover only a quarter of each head. The 16 attention layers cache about 64 KiB per token at bf16.

Dense SwiGLU FFN

No mixture of experts: every layer runs the same 17,408-dim SwiGLU feed-forward block, so all 27B parameters are used for every token.

Output head

Final RMSNorm then an untied projection to 248,320 logits. One multi-token-prediction layer drafts an extra token during decoding; it is not drawn.

Input

$0.20

per 1M tokens

Output

$2.50

per 1M tokens

Context

1M tokens

Max Output

131K tokens

Released

2026-08-14

Features

[optional-reasoning][27b-dense][vision-and-video-input][1m-tokens-context][open-weights]

Technical Specifications

LicenseApache 2.0

The Lab

location: Hangzhou, Chinafounded: 2023backed by: Alibaba Group (Alibaba Cloud Tongyi Lab)

Qwen is the language-model family of Alibaba Cloud, launched in beta in April 2023 and opened to the public that September, with Qwen-7B weights published in August 2023. Most releases through Qwen3 (28 April 2025) shipped open weights under Apache 2.0, but the Max, Plus and Flash tiers in this catalog are proprietary models served only through Alibaba Cloud Model Studio and API resellers. The current line is Qwen3.7 (Max in May 2026, Plus in June 2026), preceded by Qwen3.6 Plus and Flash in April 2026; the team's Qwen-Image models are listed on the Alibaba page.

more about Qwen →

How It Works

What vocabulary and embedding size does Qwen3.8 27B use?+

5120-dim vectors over a 248,320-token vocabulary. Qwen3.8 27B is natively vision-language: image and video tokens join this stream (the encoder is not drawn).

How many transformer layers does Qwen3.8 27B have?+

Three Gated DeltaNet layers for every Gated Attention layer (48 + 16). DeltaNet is linear attention with a fixed-size state, so only a quarter of the stack grows a cache with context.

What attention does Qwen3.8 27B use?+

What feed-forward network does Qwen3.8 27B use?+

No mixture of experts: every layer runs the same 17,408-dim SwiGLU feed-forward block, so all 27B parameters are used for every token.

How does Qwen3.8 27B produce output tokens?+

Final RMSNorm then an untied projection to 248,320 logits. One multi-token-prediction layer drafts an extra token during decoding; it is not drawn.

scene values sourced from:HuggingFace config HuggingFace config https://sebastianraschka.com/llm-architecture-gallery/#card-qwen3-8-27b

Model ID: qwen3.8-27b

Interactive Architecture

Take Qwen3.8 27B apart, layer by layer. The 3D teardown below shows its 64 layers and GQA attention, with values from the public model config.

64 layers

GQA attention · 24Q heads

27B params

Token embedding

5120-dim vectors over a 248,320-token vocabulary. Qwen3.8 27B is natively vision-language: image and video tokens join this stream (the encoder is not drawn).

64 layers, 3:1 hybrid

Three Gated DeltaNet layers for every Gated Attention layer (48 + 16). DeltaNet is linear attention with a fixed-size state, so only a quarter of the stack grows a cache with context.

Gated DeltaNet + Gated Attention

Dense SwiGLU FFN

No mixture of experts: every layer runs the same 17,408-dim SwiGLU feed-forward block, so all 27B parameters are used for every token.

Output head

Final RMSNorm then an untied projection to 248,320 logits. One multi-token-prediction layer drafts an extra token during decoding; it is not drawn.

The Lab

location: Hangzhou, Chinafounded: 2023backed by: Alibaba Group (Alibaba Cloud Tongyi Lab)

more about Qwen →

How It Works

What vocabulary and embedding size does Qwen3.8 27B use?+

5120-dim vectors over a 248,320-token vocabulary. Qwen3.8 27B is natively vision-language: image and video tokens join this stream (the encoder is not drawn).

How many transformer layers does Qwen3.8 27B have?+

Three Gated DeltaNet layers for every Gated Attention layer (48 + 16). DeltaNet is linear attention with a fixed-size state, so only a quarter of the stack grows a cache with context.

What attention does Qwen3.8 27B use?+

What feed-forward network does Qwen3.8 27B use?+

No mixture of experts: every layer runs the same 17,408-dim SwiGLU feed-forward block, so all 27B parameters are used for every token.

How does Qwen3.8 27B produce output tokens?+

Final RMSNorm then an untied projection to 248,320 logits. One multi-token-prediction layer drafts an extra token during decoding; it is not drawn.