The Kernel Take-Home: How to Approach a GPU Optimisation Exercise Under a Deadline

Kernel and performance take-homes have a recognisable shape: here is a slow program, here is the hardware, make it fast, you have a few hours. Anthropic publishes a version of this, in which candidates optimise a parallel tree-traversal workload on a simulated machine with manually managed memory, VLIW execution, SIMD and multicore features, against a published target of 1,487 cycles. NVIDIA-adjacent loops use naive GEMMs, reductions and attention variants. The details differ; the working order that succeeds does not.

Hour one is not for optimising

The instinct is to start applying techniques. Resist it, because the techniques are cheap and knowing which one matters is the entire exercise.

Get it building and get a number. Whatever the harness gives you, wrap it so you can measure a baseline reliably and repeatedly. If the exercise reports cycles, use its counter. If it does not, add timing that you trust.

Write a correctness check before you change anything. Save the naive output and compare against it after every change. This takes ten minutes and it is what stops you from discovering at hour three that a beautiful speedup has been wrong since hour two.

Compute the ceiling. This is the step that separates the exercise into "I know what to do" and "I am guessing." How many bytes does this computation fundamentally have to move, and how long does that take at the machine's memory bandwidth? How many FLOPs does it fundamentally require, and how long does that take at peak? The larger of those two is your floor, and the ratio of your baseline to that floor is your headroom.

If your baseline is at 8 percent of the bandwidth bound, you are looking for a data-movement fix, and every instruction-level idea you had is irrelevant. If it is at 70 percent of the bandwidth bound, there is at most a 1.4× left on that axis and the real win must come from moving less data, not moving it faster. Our GPU architecture track covers the roofline reasoning, and the formula sheet has the constants.

Write that ceiling into your notes immediately. It becomes the spine of the write-up.

Then change the term that binds

If you are bandwidth-bound, the wins in rough order of size: move less data (fuse operations so intermediates never reach memory, use a lower precision where the numerics allow it, exploit reuse), then move it better (coalesce accesses so a warp's loads fall in as few sectors as possible, stage through shared memory or the equivalent local store, keep enough requests in flight to saturate the pipe).

Coalescing is the highest-value single fix in most naive kernels. A strided access pattern where consecutive threads touch addresses far apart turns one transaction into many, and the fix is often a transpose of the loop indices or a change in how the data is laid out.

If you are compute-bound, look at whether you are actually using the fast path. On modern GPUs the tensor cores do the heavy matrix work and they are extremely particular about layout, alignment and shape. A kernel that silently misses them runs an order of magnitude below what the part can do. Then look at occupancy against latency hiding, but treat occupancy as a means rather than a target: more occupancy only helps if you were failing to hide latency, and it costs registers.

If you are latency-bound, which happens more often than people expect on small problems, the answer is more independent work in flight, fewer synchronisation points, and fewer launches. Kernel fusion often wins here for reasons that have nothing to do with memory traffic.

On a simulated machine with explicit memory management, as in Anthropic's exercise, the analogous moves are staging data into local memory deliberately, filling the wide instruction slots, using the SIMD width you have been given, and decomposing across cores in a way that does not serialise on a shared structure. The reasoning is identical even though the vocabulary is not.

Our CUDA and kernel questions work each of these with the arithmetic attached.

Budget the window explicitly

For a four-hour window, something like:

• 0:00 to 0:45. Build, baseline, correctness harness, ceiling computed and written down.
• 0:45 to 2:30. The one or two structural changes your ceiling analysis pointed at. Structural, not incremental: a different memory layout, a fusion, a different decomposition. Measure after each.
• 2:30 to 3:15. Incremental tuning of whatever the structure now permits. Tile sizes, unrolling, padding to kill bank conflicts.
• 3:15 to 3:45. Verify correctness properly, including edge cases and non-round sizes.
• 3:45 to 4:00. The write-up, which you have already been drafting.

Halve each block for a two-hour window and drop the incremental tuning entirely. With two hours you get one structural change, done well, and an honest account of it.

The reason to write the budget down is that the failure mode is always the same: an interesting rabbit hole at hour two eats the last two hours and the submission has no write-up.

The write-up is graded

Reported grading in this space consistently weights the explanation as heavily as the code. That is not politeness, it is the reviewer trying to work out whether you can do this again on a problem they have not chosen for you.

A good write-up is short and has five parts: the baseline measurement, the ceiling and which resource binds, what you changed and what each change bought, what you tried that did not work and why, and what you would do next with more time. That last part is where you get credit for the ideas you did not have time to implement, and candidates leave it out constantly.

Be honest about the misses. "I expected the shared memory staging to win more than it did; I think the access pattern was already partially cached, and I would confirm that with the profiler" reads as an engineer. Silence about a failed attempt reads as either luck or concealment.

On tools and rules

Read the instructions and follow them exactly. Anthropic's performance take-home explicitly permits AI assistance, which the company describes as a deliberate design choice for an evaluation meant to survive capable models, while its general candidate guidance prohibits AI unless indicated otherwise. Those two facts sit in the same policy and the difference between them is the word "indicated." Do not carry a permission across from one exercise to another, and ask the recruiter if the instructions are silent.

What this actually tests

Not whether you know a list of optimisation techniques. Everyone applying to these roles knows the list. It tests whether you can look at a program and a machine, work out which resource is the constraint, and spend limited time on the thing that moves it. That is the job, which is why the exercise looks like it.

Practise on the CUDA and kernel questions, get the roofline reasoning automatic through the GPU architecture track, and use the must-know set to find the gaps before a real deadline finds them for you.