A model you have never heard of trends on Hugging Face. Estimate the deployment before downloading it.

TL;DR: Triage in three minutes, then size in five. The go or no-go check comes first: does an engine support this exact architecture and quantization, because a model whose attention kernel or cache manager no engine implements is not deployable this week regardless of its size. Then read nine fields from config.json. Layers, hidden size and head counts give the dense shape. The expert fields give sparsity and therefore active parameters. The latent-attention fields, if present, change KV per token by tens of times. And quantization_config gives bytes per parameter, which changes the footprint by two to four times. From those, compute weights, KV per token, a KV budget at a target concurrency, and the GPU count rounded to a degree that divides the head and expert counts. Two fields change the answer by an order of magnitude and both are easy to miss: the latent-attention ranks, and the quantization format. Get either wrong and the estimate is not close.

How to approach it

Do the support check first, because it can end the exercise. Then read the config rather than the prose, since the prose rarely states what you need. Compute in a fixed order so the arithmetic is checkable. State which numbers are derived and which are assumed. Close with what you would verify by actually running it, since an estimate is a hypothesis.

A strong answer

A typical situation: a product manager asks on Monday whether a model announced over the weekend can be evaluated this week. The useful answer is a GPU count, a confidence level and a support status, and all three come from the repository page.

The triage, before any arithmetic:

go or no-go, in three checks
  1. architecture
     the config's model_type and architectures fields name the implementation
     is that architecture in a serving engine's supported list, or in a recent release note?
     new attention designs (latent, sparse indexers, hybrid linear layers) each need kernels
       and sometimes a new cache manager
  2. quantization
     quantization_config names the format
     does the engine read it, and does your hardware multiply in it natively?
     a format the hardware emulates saves memory and not time
  3. license
     is it usable for the intended purpose

if any of the three is no
  the answer is a date rather than a GPU count, and saying so on Monday is more useful than
  a sizing exercise for something that cannot run
sanity: engine support is the schedule risk on every new model, and it is the one thing that
        cannot be worked around with more hardware

Open-Weights Models of 2026 covers the landscape and what the current releases look like. Model Onboarding: From Hugging Face to Production covers what happens after the estimate.

The nine fields, and what each gives:

the arithmetic, in order
  1. weights      = total parameters x bytes per parameter
                    bytes per parameter: bf16 2.00, block-scaled FP8 1.00, MXFP4 0.53125,
                    int4 with group 128 also 0.53125
  2. KV per token = latent:  (kv_lora_rank + qk_rope_head_dim) x 2 B x layers
                    classic: 2 x layers x num_key_value_heads x head_dim x 2 B
  3. KV budget    = concurrency x tokens per sequence x KV per token
  4. total        = (weights + KV budget) x 1.15
  5. GPUs         = ceil(total / memory per GPU), rounded up to a degree dividing the head
                    count and, if using expert parallelism, the expert count

the two fields that move the answer by an order of magnitude
  latent-attention ranks: a 78-layer model with 64 heads at head_dim 192 caches 3.66 MB per
    token under classic attention and 87.75 KB with kv_lora_rank 512 and qk_rope_head_dim 64,
    a factor of 41.7
  quantization: the same parameter count is 2.00, 1.00 or 0.53 bytes each, a factor of up to
    3.8 between the extremes
sanity: getting both wrong in the same direction is a factor of over 150, which is the
        difference between one node and a cluster, so these two fields are checked twice

An example of the triage producing a fast answer:

a hypothetical release: 1.2T total, MoE with 512 experts and 8 active, latent attention with
  kv_lora_rank 576 and qk_rope_head_dim 64, 88 layers, released in MXFP4 for the experts

  weights, assuming 90% of parameters are experts
    1.2e12 x 0.90 x 0.53125 = 574 GB
    1.2e12 x 0.10 x 1.0     = 120 GB
    total                    = 694 GB
  KV per token
    (576 + 64) x 2 B x 88 = 112,640 B = 110 KB
  KV budget at 128 sequences of 16,384 tokens
    128 x 16,384 x 112,640 = 236 GB
  total = (694 + 236) x 1.15 = 1,070 GB
  on 288 GB parts: 1,070 / 288 = 3.7 -> 4 GPUs, and check that 4 divides the head count
sanity: the whole estimate is five lines of arithmetic and it answers the product question,
        with the caveat that the 90% expert share is an assumption and the engine support
        check is the real gate

The checks that turn the estimate into a verified number, once a replica is up:

verifying each derived quantity against the engine
  weights        the engine logs the loaded model size; compare against
                 total parameters x bytes per parameter
                 a 2x discrepancy means quantization_config was not applied
  KV per token   the engine reports its KV cache size in blocks and tokens
                 tokens_reported x your KV-per-token should equal the pool bytes
  active params  measure instead of deriving: at batch 1, tokens/s x active bytes should
                 approach the aggregate memory bandwidth
  parallel degree  nvidia-smi topo -m to confirm NVLink is present, since a tensor-parallel
                 deployment on a fabric-manager-less system runs and is slow

useful while doing it
  nvidia-smi -q -d MEMORY per rank during load, to see the real per-GPU weight share
  NCCL_DEBUG=INFO on the first collective, to confirm the topology the engine detected
sanity: the KV pool cross-check is the strongest single verification, because it depends on
        the layer count, the attention design and the dtype all being read correctly

The reversal condition: if the model is dense rather than a mixture of experts, most of this simplifies and the estimate gets easier and less interesting. Weights are the parameter count times bytes per parameter with no expert share to assume, active parameters equal total parameters so decode throughput follows directly from bandwidth, and the KV calculation is whatever the attention design says. The reason the sparse case needs care is precisely that memory and throughput have decoupled, and applying dense intuition to a sparse model gives an answer that is wrong in both directions at once: too pessimistic on speed and too optimistic on the number of GPUs.

What interviewers probe next

• "What is your confidence?" Weights are exact if the parameter count and format are stated; the expert share is an assumption; the KV budget depends on a concurrency you chose. Say which is which.
• "How do you check engine support quickly?" The engine's release notes and any launch note for the model, then a single-replica bring-up, which is the only real test.
• "What if quantization_config is absent?" Then the weights are in whatever dtype says, usually bf16, and quantizing is your project with your own evaluation.
• "What would you verify first?" The engine's reported KV pool against your prediction, because a mismatch means a config field was read wrong.

Common mistakes

• Sizing before checking engine support, which is the constraint that cannot be solved with hardware.
• Computing KV from head counts when latent-attention fields are present.
• Ignoring quantization_config and assuming bf16, which doubles or quadruples the footprint.
• Confusing total and active parameters, so memory and throughput estimates are swapped.
• Presenting the estimate without saying which numbers are derived and which are assumed.

Key takeaways

• Check architecture support, quantization support and license before any arithmetic; that gate is a date rather than a GPU count.
• Nine config fields carry the sizing, and two of them move the answer by an order of magnitude: the latent-attention ranks and the quantization format.
• Bytes per parameter: bf16 2.00, block-scaled FP8 1.00, MXFP4 and group-128 int4 both 0.53125.
• Compute weights, KV per token, KV budget at a stated concurrency, total times 1.15, then round to a degree dividing the head and expert counts.
• Getting both order-of-magnitude fields wrong compounds to over 150 times, which is the difference between one node and a cluster.