mediumNewNVIDIACerebras

Write a thread-safe counter in C++ that many threads increment. What are the memory-ordering and layout choices?

One atomic increment is correct and does not scale, because every thread fights for the same cache line. The ordering that is sufficient and why the default is stronger than needed, the padding that removes contention entirely, and the arithmetic showing when sharding is worth it.

Updated Sep 2026 · Grounded in real AI infrastructure interview loops and written to a senior-engineer editorial bar, with every number worked and every diagram hand-built.

20 answers per topic instead of 10, plus saved progress and bookmarks · no cardor unlock all 283 remaining answers · ₹2,000 / $25

The concepts behind this question

Ranked by how closely each one overlaps this question's topic, so the first card is the thing to read if the answer above moved too fast.

Foundational

💻 Coding for Infra

Cache-Friendly Data StructuresA cache line is 64 bytes and it is the unit of coherence, so where data sits decides how fast code runs more often than which algorithm it uses. Two consequences dominate infrastructure code: a lookup that chases a pointer pays two dependent memory stalls instead of one, and two threads updating adjacent variables contend for a line they do not logically share. Both are layout problems with layout fixes.

Advanced

💻 Coding for Infra🔒 Premium

Concurrency in Python, Go and C++Infrastructure code is concurrent by nature: a loader feeding a GPU, a gateway holding ten thousand streams, a controller reconciling a fleet. The coding screen tests whether you know which primitive fits which problem in the language you claim, and the three languages the field uses answer differently: Python has one interpreter lock and an event loop, Go has cheap goroutines and channels, C++ has threads, mutexes and atomics with no safety net. This page gives the model of each, works the favourite problems (a thread-safe LRU, a worker pool, a bounded fan-out) in each, and derives when threads, processes or async buy throughput.

Foundational

🧮 Napkin Math & Capacity

KV Cache SizingThe KV cache is the memory that decides how many users a serving replica can hold and how long their context can be. Its size per token comes from four numbers in the model's config file (layers, KV heads, head dimension, bytes per element) and one formula; multiplied by context and concurrency it is the number every capacity plan is built on. This page derives it, works it for four models including an MLA one, and shows the two places candidates get it wrong by a factor of eight.

Foundational

💻 Coding for Infra

The GPU Credit Scheduler PatternThe most widely reported coding problem in AI infrastructure loops is a small scheduler: accounts hold credits, jobs arrive with a cost and a priority, and you must decide which jobs run, in what order, without letting any account overspend, then extend it under follow-ups (refunds, reservations, concurrency limits, fairness). It is not a trick question; it is a test of whether you can model state cleanly, pick the right data structures, keep invariants under mutation, and talk about complexity while typing. This page works the problem from the first line to the fourth follow-up, with the code, the invariants, and the derivations.

Explore the full curriculum →

UP NEXT ON YOUR JOURNEY

Next in this trackWhy is std::unordered_map rarely the fastest hash map, and what would you use instead?Next in this trackImplement a work-stealing deque. Why do the owner and the thieves take from opposite ends?Next in this trackSimulate speculative decoding to find the expected tokens per round and the batch size where it stops paying.

FEDITOR'S NOTE

Scored on relaxed ordering being sufficient for a counter, on false sharing and cache-line padding, and on the sharded design with the read cost that it trades for.

DISCUSSION · 0

No comments yet — be the first to share your approach.