hard★ EssentialNewCoreWeaveLambdaOpenAI

Should we buy or rent 1,000 H100s for the next two years?

Capex plus two years of power and operations, minus what the cards are worth at the end, against a rental rate at a utilization. The chain that produces a break-even utilization, and the two inputs that swing it from 60% to over 100%.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

more free answers with an account · no card

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

TCO: Buy vs RentWhether to buy GPUs or rent them is a utilization question dressed as a finance question. An owned H100 costs a few tens of thousands of dollars up front and a known amount per hour in power, cooling, space and operations; a rented one costs a few dollars per hour and nothing when idle. The break-even is the utilization at which the owned hourly cost, amortized over the hardware's useful life, equals the rental rate. This page builds the owned cost from parts, works the break-even, and adds the terms the simple model leaves out: depreciation risk, reserved discounts, and the price of idle capacity.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Cost per Million TokensThe unit every serving decision cashes out in. It is one formula: the fleet's dollars per second divided by the tokens per second it sustains, scaled to a million, with utilization in the denominator because idle replicas still cost money. This page derives it from a GPU price and a throughput estimate, works it at three batch sizes to show why batching is the main lever, separates prefill from decode pricing, and shows how the same fleet's cost per token moves by 5x between a quiet hour and a busy one.

Advanced

🧮 Napkin Math & Capacity🔒 Premium

Capacity Planning and UtilizationCapacity planning for GPUs is deciding how many to have next quarter given that they cost money whether busy or not, that demand arrives in bursts, and that a queue near saturation produces waits that grow without bound. This page works the planning arithmetic for a serving fleet (peak demand, headroom, the p99 penalty of running hot) and a training platform (job mix, queue wait, the value of a shared pool), and gives the queueing intuition that makes 70% look full. The number that decides everything is utilization, and it has a ceiling set by latency, not by hardware.

Foundational

🧭 Ownership & Judgment

Talking About Cost and Capacity with LeadershipInfrastructure engineers are asked to justify large numbers to people who do not share their vocabulary, and the conversations go wrong in predictable ways: a technical objection with no alternative, a forecast with no assumptions, or a cost quoted in a unit the listener cannot act on. What works is a small number of costed options, a stated recommendation, the decision needed by a date, and every figure expressed in whatever the listener actually controls.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackHow much does moving from bf16 to fp8 save in serving cost?Next in this trackHow long does an all-reduce of a 70B model's gradients take on eight GPUs?Next in this trackHow many tokens per second can one B200 decode for a 70B model?

FEDITOR'S NOTE

Scored on whether the candidate turns the question into a break-even utilization with named assumptions, and then says which assumption (residual value, rental rate) the answer is most sensitive to. A flat 'buying is cheaper' is the failing answer.

DISCUSSION · 0

No comments yet — be the first to share your approach.