GLM-5.3 was released this morning. How many GPUs do you need to serve it, and of what kind?

TL;DR: Five steps from the config. As of September 2026 Z.ai publishes GLM-5.3 at 753B total parameters with weights released in FP8 e4m3 and 128 by 128 block scaling, so weights are about 753 GB plus roughly one percent for scales, call it 760 GB. Its config.json shows latent attention with kv_lora_rank 512 and qk_rope_head_dim 64 over 78 layers, so the KV cache is (512 + 64) times 2 bytes times 78, which is 89,856 bytes or 87.75 KB per token, roughly a quarter of the corpus's 320 KB figure for a 70B grouped-query model despite ten times the parameters. Pick a target concurrency, add the KV, add about 15 percent for activations and fragmentation, and divide by the memory per GPU. On 288 GB parts that lands at 8 GPUs; on 180 GB parts at 16; on 80 GB parts the weights alone need 16 before any cache. Then check the degree divides the 64 attention heads and the 256 experts, which 8, 16 and 32 all do.

How to approach it

Read the config before the model card's prose, because the numbers are there. Compute weights, then KV per token, then the KV budget at a stated concurrency, then the total with overhead, then the GPU count rounded to a valid parallel degree. Say which of those numbers you are least confident about. Close with the launch line and what you would measure first.

A strong answer

A typical situation: a model lands at 9 a.m. and a product team asks whether it can be serving by the afternoon. The answer depends on the footprint and on whether an engine supports its attention design, and both are answerable from the repository in minutes.

The derivation:

from the published config.json, checked September 2026
  num_hidden_layers        78 (plus 1 multi-token-prediction layer)
  hidden_size              6,144
  num_attention_heads      64
  kv_lora_rank             512
  qk_rope_head_dim         64
  n_routed_experts         256, num_experts_per_tok 8, n_shared_experts 1
  quantization_config      fp8, e4m3, weight_block_size 128 x 128
  max_position_embeddings  1,048,576
  total parameters         753B, from the model card

step 1: weights
  753e9 x 1 byte = 753 GB, plus about 1% for block scales = about 760 GB

step 2: KV per token
  latent attention, so the cache is the compressed vector plus the rotary part:
  (512 + 64) x 2 B x 78 layers = 89,856 B = 87.75 KB per token
  NOT 2 x 78 x 64 x 192 x 2 B = 3.66 MB, which is what assuming classic attention would give
  the ratio between those two is 41.7, which is the size of the mistake

step 3: KV budget at a target concurrency
  target 256 concurrent sequences at 32,768 tokens
  256 x 32,768 x 89,856 = 753 GB

step 4: total
  (760 + 753) x 1.15 for activations, workspace and fragmentation = 1,740 GB

step 5: GPUs, rounded to a valid degree
  288 GB parts: 1,740 / 288 = 6.04 -> 8
  180 GB parts: 1,740 / 180 = 9.67 -> 16
  80 GB parts:  1,740 / 80  = 21.8 -> 24 or 32; and weights alone are 760 / 80 = 9.5 -> 16
  valid degrees must divide 64 heads: 1, 2, 4, 8, 16, 32, 64
  and 256 experts divides by any power of two, so expert parallelism is unconstrained here
sanity: the KV budget equals the weight footprint at this concurrency, which is the regime
        these models are designed for, so a sizing that counts only weights is short by about
        half

Reading config.json to Size a Model You Have Never Run covers the method. Multi-Head Latent Attention and Sparse Indexers covers why the KV is so much smaller than the parameter count suggests.

What changes the answer:

the launch, once the sizing is settled
  vllm serve zai-org/GLM-5.3 \
    --tensor-parallel-size 8 \
    --max-model-len 32768 \
    --max-num-seqs 256 \
    --gpu-memory-utilization 0.92 \
    --enable-expert-parallel

  what to check immediately
    the engine's reported KV cache size in blocks and tokens, against the 13.4M tokens the
      arithmetic predicts at this configuration
    a two-request smoke test comparing output against the model card's own examples
    every eos_token_id from the config is configured; this model lists three
sanity: comparing the engine's own reported cache size against your prediction is the check
        that the model was understood, and a large discrepancy means one of the config fields
        was read wrong

The reversal condition: if the product's context requirement is much smaller than 32,768, the whole sizing shifts and a much cheaper deployment works. At 4,096 tokens per sequence the KV budget for 256 sequences is 256 times 4,096 times 89,856, which is 94 GB rather than 753 GB, and the total falls to about 982 GB, which is 4 parts of 288 GB. The single largest lever in this exercise is the context length you actually need, and taking the model's maximum because it is in the config is how a deployment ends up three times larger than the product requires.

What interviewers probe next

• "What if you assumed classic attention?" You would compute 3.66 MB per token instead of 87.75 KB and conclude the model needs tens of times more memory, which is the most common error on this class of model.
• "Why round to 8 rather than 7?" The tensor parallel degree must divide the attention head count, and 64 divided by 7 is not an integer.
• "Where would you look for engine support?" The engine's release notes and the model's own launch note. Support for the model family is not the same as support for this model's attention and quantization combination.
• "What is the first thing to measure?" The engine's reported KV pool against your prediction, then a real concurrency sweep rather than a single-request latency number.

Common mistakes

• Computing KV from heads and head dimension when latent attention fields are present, which overestimates by 41.7 times here.
• Sizing from weights only, when the KV budget at realistic concurrency is comparable.
• Using the model's maximum context rather than the product's requirement.
• Choosing a parallel degree that does not divide the head count, which pads and wastes memory.
• Ignoring quantization_config and sizing FP8 weights as bf16, which doubles the GPU count for no reason.

Key takeaways

• GLM-5.3 is 753B in FP8, so about 760 GB of weights including block scales.
• KV per token is (kv_lora_rank + qk_rope_head_dim) x 2 B x layers = 87.75 KB, not the 3.66 MB classic attention would give.
• At 256 sequences of 32,768 tokens the KV budget is 753 GB, roughly equal to the weights.
• Total about 1,740 GB, so 8 parts at 288 GB, 16 at 180 GB, and at least 16 at 80 GB before any cache.
• The context length you actually need is the largest lever: 4,096 instead of 32,768 cuts the deployment to about 4 GPUs.