Research

Qwen3.5 vs Qwen3: Day-One Benchmarks on Self-Hosted Inference

Qwen3.5-35B-A3B dropped and we ran our full production benchmark suite against it the same day. Quality matches our current model at 11/11. Throughput is roughly half. The reason is interesting, and it tells you exactly when to switch.

February 2026 | Dry Ground AI Research

Key Findings

Five things we learned in 24 hours

We ran Qwen3.5 the same day it became available. Here is what we found before the community had time to catch up on quantized variants.

Quality is identical

Both models score 11/11 on our production benchmark suite covering JSON extraction, email classification, summarization, instruction following, reasoning, code generation, and email drafting. Qwen3.5 does not improve quality on these tasks.

Throughput gap is a quantization story

The ~50% throughput difference is entirely explained by BF16 vs GPTQ-Int4 precision. Qwen3.5's 16.6 tok/s vs Qwen3's 31.8 tok/s is a quantization comparison, not a model comparison. When GPTQ quantization becomes available for Qwen3.5, we expect parity or better.

Architecture is a generational step forward

Qwen3.5 uses Gated Delta Networks (a hybrid attention/SSM design). It supports 1M native context vs Qwen3's 32K, improves reasoning under the hood, and community testing suggests better quantization resilience. That matters for what comes next.

Classification efficiency held

Qwen3.5 with thinking disabled produces 2-3 token classifications, same as Qwen3. Email triage latency was 468-667ms vs 309-675ms for Qwen3. Comparable, not worse. The structured output behavior transferred cleanly.

BF16 fits on a single 80GB GPU, barely

Qwen3.5 uses 78GB of an 80GB GPU at BF16. That leaves minimal headroom for KV cache, which limits concurrency. Production deployment requires GPTQ-Int4 for anything beyond single-user workloads.

Benchmarks

The numbers at a glance

Tested February 25, 2026. Qwen3 running GPTQ-Int4 on production hardware. Qwen3.5 running BF16 on the same hardware (78GB of an 80GB GPU).

11/11

Quality Score

Qwen3.5 passes all tests (100%)

31.8

Qwen3 Throughput

tok/s long generation (GPTQ-Int4)

16.6

Qwen3.5 Throughput

tok/s long generation (BF16)

Context Window

native tokens in Qwen3.5 vs 32K

~16 GB

VRAM (Qwen3)

GPTQ-Int4 on dedicated GPU

~78 GB

VRAM (Qwen3.5)

BF16 on dedicated GPU (80GB)

Performance

Head-to-head latency and throughput

All latencies include approximately 50-100ms network overhead from our RunPod HTTPS proxy. Five iterations per prompt size, temperature set to 0.0.

Single-request latency

Prompt Size	Qwen3 (GPTQ-Int4)	Qwen3.5 (BF16)	Delta
Short (16 tok)	448ms / 5.1 tok/s	458ms / 4.7 tok/s	-8% tok/s
Medium (128 tok)	3,668ms / 28.1 tok/s	6,555ms / 15.7 tok/s	-44% tok/s
Long (700 tok)	22,023ms / 31.8 tok/s	42,095ms / 16.6 tok/s	-48% tok/s

Concurrency scaling (aggregate tok/s)

Concurrent Streams	Qwen3 (GPTQ-Int4)	Qwen3.5 (BF16)	Delta
1	28.6	15.6	-45%
2	56.5	28	-50%
4	114.1	54.8	-52%
8	220.8	106.6	-52%

Scaling efficiency comparison: Qwen3 scales from 1 to 8 concurrent streams with 96% efficiency (7.7x throughput increase). Qwen3.5 at BF16 hits 85% efficiency (6.8x increase). Both are strong. The gap at 8 concurrent streams stays at roughly 2x total throughput, consistent with the quantization difference.

Quality

11 tests, both models, full results

Both models pass every test. The latency column tells the real story: Qwen3.5 is slower on most tasks, but the margin varies significantly by task type. Code generation is the standout exception.

Test	Description	Qwen3	Qwen3.5	Qwen3 Lat.	Qwen3.5 Lat.
json-extract-1	Contact info to JSON	✓	✓	2,032ms	7,647ms
json-extract-2	Meeting notes to JSON	✓	✓	3,512ms	8,097ms
classify-email-1	Urgency classification	✓	✓	475ms	667ms
classify-email-2	Newsletter classification	✓	✓	309ms	468ms
classify-email-3	Action required	✓	✓	675ms	565ms
summarize-1	Financial summary	✓	✓	2,388ms	4,445ms
instruct-1	Format following (5 bullets)	✓	✓	3,597ms	8,973ms
instruct-2	Arithmetic (strict format)	✓	✓	619ms	1,065ms
reason-1	Capacity calculation	✓	✓	411ms	863ms
code-1	Python function gen	✓	✓	8,686ms	8,528ms
email-draft-1	Concise email draft	✓	✓	1,459ms	2,101ms

Code generation anomaly: The one place Qwen3.5 is nearly identical in speed is code generation (8,686ms vs 8,528ms). This suggests that at the token-by-token generation level, the two models are comparable for complex output. The throughput difference in other tasks is largely from BF16 batch efficiency, not per-token capability.

Architecture

What changed from Qwen3 to Qwen3.5

Qwen3.5 is not a scaled-up Qwen3. The attention mechanism is fundamentally different. Gated Delta Networks combine standard attention with state space models (SSM), which is the same family as Mamba/H3 architectures. The practical result: 1M context without the memory explosion that kills standard transformers at long ranges.

1M context window

Native, not extended via tricks. For workloads that need to process entire document sets in one pass, this changes what is architecturally possible.

Better quantization resilience

Community testing suggests Qwen3.5 degrades less under aggressive quantization than standard transformer MoE. If confirmed at GPTQ-Int4, the throughput gap closes and Qwen3.5 becomes the clear upgrade.

Same active parameter budget

3B active parameters per token, same as Qwen3. The MoE routing overhead is identical. This is why, once quantized, we expect similar VRAM and throughput to the current production model.

Feature	Qwen3-30B-A3B	Qwen3.5-35B-A3B
Attention mechanism	Standard MoE	Gated Delta Networks (hybrid attention/SSM)
Total parameters	30B	35B
Active parameters per token	3B	3B
Max context	32,768 tokens	1,048,576 tokens (1M)
Quantization resilience	Good	Very resilient (community-reported)
VRAM at BF16	~75 GB	~78 GB
VRAM at GPTQ-Int4	~16 GB	TBD (quant pending)
License	Apache 2.0	Apache 2.0

Production

What we are running and when we plan to switch

Now

Current: Qwen3-30B-A3B (GPTQ-Int4)

Identical quality, nearly 2x the throughput. Serves 16 concurrent streams with minimal per-user degradation. Production stays where it is until there is a measurable reason to move.

1.GPTQ-Int4 quantization available (expected within 1-2 weeks as the community catches up). This is the throughput unlock.
2.Our benchmark suite passes 11/11 on the quantized model. Same bar as the current model.
3.Throughput matches or exceeds 28+ tok/s on medium generation. We do not accept a regression in throughput without a quality gain to justify it.

Migration path

Wait for GPTQ-Int4 quant of Qwen3.5-35B-A3B to become available

Run our full 11-test quality suite against the quantized model

If quality holds at 11/11 and throughput meets bar: swap production

Keep Qwen3 weights on disk as rollback (takes about 2 minutes to swap)

Methodology

How we ran this benchmark

We ran both models on the same hardware, same day, with the same test suite we use for every weekly benchmark run.

Date

February 25, 2026

Hardware

Single dedicated GPU (80GB PCIe)

Qwen3 framework

vLLM 0.15.1

Qwen3.5 framework

vLLM 0.16.0rc2 nightly

Qwen3 flags

--enforce-eager --max-model-len 4096 --quantization gptq --dtype float16

Qwen3.5 flags

--enforce-eager --max-model-len 4096 --gpu-memory-utilization 0.95

Thinking (Qwen3.5)

Disabled via chat_template_kwargs (enable_thinking: false)

Temperature

0.0 for all quality tests

Performance iterations

5 per prompt size

Concurrency levels

1, 2, 4, 8 simultaneous streams

Network overhead

RunPod HTTPS proxy (~50-100ms per request)

Quality suite

11 tests across 7 categories: structured output (JSON extraction from contacts and meeting notes), email classification (urgency, newsletter, action-required), summarization (financial documents), instruction following (format and constraint compliance), reasoning (multi-step calculation), code generation (Python function), and email drafting. Each test has explicit pass/fail criteria based on output correctness and format.

Why day-one benchmarks matter

When a new model releases, the community needs a few weeks to produce quantized variants and run independent evals. Day-one data at BF16 gives you a quality signal and an architecture read. The throughput numbers are a comparison point, not a final verdict. We will re-run this suite when quantized Qwen3.5 is available.

Quantization is the throughput story

The 50% throughput gap between Qwen3.5 and Qwen3 is not a model capability difference. It is a precision difference. Qwen3 at GPTQ-Int4 uses roughly 16GB VRAM. Qwen3.5 at BF16 uses 78GB. The same GPU is doing 5x more memory work per token for Qwen3.5.

When GPTQ-Int4 quantization becomes available for Qwen3.5, VRAM drops to a projected 18-20GB (35B model vs 30B, slightly more than current). The throughput gap closes. If the architecture-level quantization resilience community testing suggests holds, Qwen3.5 may actually beat Qwen3 at the same precision level.

This is the benchmark you run at day one to know what to expect at day fourteen. The quality signal is clear. The throughput signal is pending.

Read the full self-hosted inference story

This benchmark is a comparison between two specific models. For the broader picture (why we self-host, how we got to our current production stack, and what we learned across three quantization configurations) see the full February 2026 research article.

State of Self-Hosted Inference: February 2026

Benchmarks conducted February 25, 2026 on dedicated GPU infrastructure (80GB VRAM). Qwen3 tested at GPTQ-Int4, Qwen3.5 tested at BF16 (quantized variants not yet available at time of benchmark). Quality suite: 11 tests across 7 categories under production conditions. Network latencies include RunPod HTTPS proxy overhead (~50-100ms). Concurrency tested at 1, 2, 4, and 8 simultaneous streams.

We use cookies to improve your experience. Cookie Policy · Privacy Policy