The Board Is the Only Benchmark: 72 Hours of Speed Demon 2026

Team ttimesai on four tracks in zero-dependency Rust: hashes instead of byte compares, memory you pay for without noticing, profiling through a file name, and a bug that showed up in 12 runs out of 30.

Speed Demon 2026, run by Hackathon Raptors, gave teams 72 hours (30 October 15:00 UTC to 2 November 15:00 UTC) to beat naive Python references on four tracks: A · Ingest (aggregate 3.1M log lines into per-endpoint, per-hour percentiles), B · Index (50,000 boolean and phrase queries over a 250 MB corpus), C · Join (parse a hostile multi-megabyte CSV and join it against regions), and D · Traverse (2,000 shortest-path queries on a 5M-edge weighted graph). The speed score per track is 50 · log10(K), where K is the product of four ratios against the reference, with exponents for wall time (0.50), peak memory (0.20), source lines (0.15) and dependencies (0.15); the "points" below are this speed score, and the track total adds the judged parts on top. The live board runs every push once on a 12-thread i5-13500 with turbo off.

Our repo: https://github.com/ttimesai-star/speed-demon (public from 2 November, after code freeze). Every number below comes from the board's own runs or from the PERF.md logs in that repo. Board results are from the public board on 31 October; the final scoring (five runs of the frozen commit on a hidden seed, on a machine with no other load) is run by the organisers and published with the winners on 13 November, after this post, so it is not in here.

How the code was written. Plainly: the code was written by Claude Code, Anthropic's AI coding assistant, in long sessions, and a human on the team set the goals and the rules. The assistant also read the board results, decided which experiments to run, and wrote this post from the repo logs. The rules it worked under were: check output hashes before every push, keep at least 15 minutes between pushes, trust only the board for speed and memory, and log failures as well as wins. Rule 06 of the hackathon allows this ("we do not gatekeep on how the code was produced"), and every track README says it. Rule 06 also asks that PERF.md shows a human understood what happened; the log in the repo is the evidence for that, and you can judge it. Every figure below was checked against the repo logs.

Where we ended up (public board, 31 October)

Our three best tracks summed to 225.8, and the board's overall ranking, which counts the best three tracks, had us 5th of 99 teams on 31 October. For scale, the reference takes 32.05 s on track A. The final scoring is the organisers' and comes later, as noted above.

1. You cannot benchmark this on your laptop

The first scored run of track A was the lesson that shaped the rest of the weekend. On our 2-vCPU CI runner the program peaked at 18.5 MB. The board measured 730.7 MB and 0.132 s (run-299ca5c2b1ad).

The cause was our own reservations: 12 threads × 72 hours × a "worst-case" vector. Our best explanation is that the board machine has transparent huge pages on and backed those reservations with 2 MB pages. Every touched huge page counted in full, and zeroing them ate most of the wall time. We could not reproduce the exact 730 MB in CI (with THP=always and 12 forced threads we got 55.5 MB), so that part is inference, not proof. The fix was to reserve for the mean hour and opt out of THP at start. Later we also pinned glibc's mmap threshold:

unsafe { prctl(41, 1u64, 0u64, 0u64, 0u64) }; // PR_SET_THP_DISABLE: small vectors, no 2 MB pages unsafe { mallopt(-3, 1 << 16) }; // M_MMAP_THRESHOLD fixed at 64 KB: a freed hour's records leave M at once 

The first line took A from 0.132 s / 730.7 MB to 0.0340 s / 29.8 MB in one push (run-ffe31d35df4b). The second line is subtler. glibc raises its mmap threshold after each large free, so the per-hour buffers we freed during output stayed in the arena and in peak memory. With the threshold fixed, M dropped from 29.0 to 26.8 MB at the same T (run-2e4a68da0bef). Track D got the same line later.

Our two test machines disagreed with the board in both directions:

• Track D, hash tables instead of dense label arrays: CI was 4 % slower, the board 2.2× slower (run-a741c2a56a3e). Reverted.
• Track B, a Bloom filter in front of the dictionary: PC −20 %, CI +2 %, board no change (run-9aa73a6479ad). Reverted.
• Track B, PGO: board 0.6456 s against 0.6115 s without it (run-a6a09a082994). Reverted.
• Track C, 32-byte inline name slots: slower on our PC, whose 96 MB L3 holds everything, and faster on the board's 24 MB L3 (run-20344632fbda, kept).

From ~20:50 UTC on day one we also had no CI: the private repo had used up its GitHub Actions minutes on A/B runs. From then on we checked correctness on a Windows PC build of the same logic and measured speed and memory only on the board.

2. Profiling a machine you can't log into

We needed phase timings from the board itself, but it only reports T, M and a short public failure string. So a profiling build writes its timings into the name of an extra empty file in /out:

let t0 = std::time::Instant::now(); let us = || t0.elapsed().as_micros(); // ... at the end of main: let name = format!("A_nt{nt}_l{tl}_b{tp}_s{}-{}_e{}-{}-{}_j{tj}_g{tg}_w{tw}_f{}-{}-{}_x{}", st[0], st[nt - 1], en[0], en[nt / 2], en[nt - 1], fin[0], fin[nt / 2], fin[nt - 1], us()); fs::write(format!("{dout}/{name}"), b"").unwrap(); 

The run then fails on purpose ("/out must hold exactly …, found […]"). That failure text is public on the board, and T is still recorded. We checked separately on the PC that the real output was correct. These runs were measurements only and never counted toward a best score.

The first profile of A (run-5db9c3f80e78) showed: parse ends at 19.3 / 19.6 / 19.9 ms (first / median / last thread), output workers finish around 29.9 ms, and main returns at 29.97 ms of a 33.5 ms T. ~3.5 ms was process start and exit, mostly unmapping the 239 MB input in one thread. So each parse thread now drops its own pages as soon as it finishes:

// MADV_DONTNEED on the whole pages of d[a..b]: unmapped here in parallel, not at the exit in one thread let (p, e) = (a.next_multiple_of(4096).min(b), b / 4096 * 4096); unsafe { madvise(d.as_ptr().add(p.min(e)), e - p.min(e), 4) }; 

A control push without it ran 0.0356 s, against 0.0329 s with it (run-8c3d925f7fbd and run-3a49e8d76354), so it is worth about 2.5 ms. That is about 7 % of the program, for code that computes nothing.

The same trick works for ablations. On track C we pushed a build that skips setting the per-country bitset bits. The output is wrong on purpose, but the run is timed: T 0.0523 s, M 9.8 MB (run-138c8acedbb3). So the bitset costs about a tenth of T and 3.7 MB. That is less than per-thread buffers would cost in memory points, so it stayed.

3. The biggest single lever: stop comparing key bytes

The largest single jump of the second day came from one idea on two tracks. Identify a key by a 64-bit hash and never compare its bytes. On C, a perf profile of the first version had 23 % of the time in bcmp, comparing each name with its first occurrence at a random spot in a 202 MB mapping. Inline slots and arenas helped. Then we dropped the bytes entirely:

// Dense id of the name with hash h, shared by all threads. A slot holds the low 44 bits of h << 20 | id and its // index is the top 17 bits of h, so names count as equal when all 61 bits match: with 100,000 names the chance // that any two do is about 100,000^2 / 2^62, below 10^-8. An 8-byte slot per name keeps the table at 1 MB. fn name_id(t: &[AtomicU64], h: u64) -> u64 { let (mut i, tag) = ((h >> 47) as usize, h & (1 << 44) - 1); loop { let mut v = t[i].load(Acquire); if v == 0 { let id = TOP.fetch_add(1, Relaxed); match t[i].compare_exchange(0, tag << 20 | id, Release, Acquire) { Ok(_) => return id, Err(w) => v = w, } } if v >> 20 == tag { return v & 0xFFFFF; } i = (i + 1) & (SLOTS - 1); } } 

No more random reads into the input to compare bytes, and the table shrank from 4 MB to 1 MB. Our profile had already shown C running 3.3× slower per thread on the board than on our PC, which we put down to cache misses on tables of ~9 MB. T went from 0.0481 to 0.0416 s, M from 13.4 to 10.5 MB: +2.7 points in one push (run-c21f5e3b4e84). On A the same change for endpoints gave 0.0294 s and +0.6 points (run-3b77efb47584). There, two of the 1,842 templates collide with a probability below 10⁻¹².

This trades exactness for a stated, tiny probability, and we documented the bound in PERF.md and the README. Every local dataset (published hashes plus the organisers' reference run on extra seeds) still matched byte for byte.

4. The bug that full-size runs never showed

That name_id function had a bug. The empty name hashes to 0. If it happened to be the first name inserted, it got id 0, and its slot then held exactly 0, which reads as "empty". The next empty name would claim a fresh id. Whether this happened depended on which thread got to the table first, so the output depended on thread timing.

Five runs of the full dataset all passed, and so did the board. The spec's tiny worked example, run 30 times, was wrong 12 times. On full data we estimated about one run in a hundred would fail. The final score is five runs on a hidden seed, so this could have cost us the whole track. The fix was one character:

-static TOP: AtomicU64 = AtomicU64::new(0); // names (a lost race for a slot leaves an unused id) +static TOP: AtomicU64 = AtomicU64::new(1); // names from 1, so no slot is 0 (the empty name hashes to 0); a lost race leaves an unused id 

After the fix: 60/60 worked-example runs, 30/30 tiny, 20/20 sample, and all ten datasets matched the reference (commit ebb2a44). From then on every candidate version went through 30 repeated runs of each small dataset, not only the big ones. Small inputs make rare orderings likely. Big inputs hide them.

5. Memory on the graph track: 279 MB to 79 MB

Track D started at 279 MB peak. That was mostly 12 threads × dense label arrays plus a graph stored as 8-byte edges. The leaders were at 70–110 MB. The steps that got us there:

• Free early. The parsed edge buckets are freed right after the first CSR, and the reverse CSR is built inside the old edge array: M 279 → 139 MB (run-783c571ed206).
• Paged labels. Each thread keeps a pool of small pages of [u32; 2] labels instead of n labels per thread.
• Pack the edges into 4 bytes. At full scale a node id needs 21 bits, which leaves 11 for the weight, and 99.93 % of weights fit:

// A search graph: offsets, edges as head | weight << .3 (the head takes .3 bits; a weight that does not fit // leaves the field's top value and is kept exact in .2, sorted by edge position). Weights are mostly below // 2^11 and n is below 2^21 at full scale, so an edge is 4 bytes instead of 8. struct G(Vec<u32>, Vec<u32>, Vec<(u32, u32)>, u32); 

That cut M from 137 to 90 MB at the same T (run-a4daf0dfb2bb). A radix heap, prefetches in the BFS renumbering, and running queries in order of their source's new id brought T from 0.194 to 0.176 s. Final: 0.1757 s, 79.3 MB (run-d27164d64d22).

6. Where track B stalled

Track B never caught up: 0.43 s against about 0.27 s for the track leader on the night of 30 October. The cost sat in tokenizing and dictionary lookups. Only memory responded: the peak was held by 75,000 small allocations and per-thread term streams. One shared list buffer, u16 term ranks in the streams, and dropping the 75,000 offset vectors took M from 185 to 154 MB, and a later step reached 150 MB. B stayed out of our best three.

7. Noise: ±5–10 % on unchanged code

Track A ran anywhere from 0.029 to 0.041 s on identical code across the day. One push of a compacted track C read 0.0651 s; the same code pushed again read 0.0481 s. A D outlier of 0.1922 s re-measured at 0.1778 s. What we did about it:

• Control in every push. The board re-runs every track on each push, and track B was unchanged for the whole second day, so its time showed how fast the board was in that hour. When an unchanged track ran 12 % slow in the same push as an experiment, we did not trust that push's verdict either way.
• Compare within one push, never across hours.
• Re-measure anything under ~5 % before believing it. Several "wins" were noise: prefetchw and an AVX2 byte search on C, and a compact key arena on A, all ended up inside the spread of unchanged code.
• Pick finals by mean, not best. The board keeps your best run, but the final scoring takes five runs of the frozen commit on a hidden seed on a quiet machine. So pushing the same code again for a luckier number buys nothing. We chose final versions by the average of several board runs (A ≈ 0.0303 s, C ≈ 0.041 s, D ≈ 0.177 s).

8. Source lines are points too

S has a weight of 0.15, counted after rustfmt. Rewriting the same algorithm more compactly took C from 368 to 288 lines and D from 429 to 363, worth +0.5 to +0.8 points with no change in speed. The other side: 30 extra lines cost about as many points as 2–3 % of T, so a small speed tweak can lose more than it wins. We counted lines with the board's formatter rules before every push.

What we would tell the next team

1. Get on the board in the first hour with something trivial. Our CI numbers predicted little about the board.
2. Build a profiling channel into the board (a file name, an exit code, anything public) before optimising.
3. Make serial work your first target. With 12 threads waiting, a one-thread sort of 200k country codes cost track C about a quarter of its time.
4. Hash identities beat byte compares, if you state the collision bound and check every dataset you have.
5. Repeat small inputs many times. Our worst bug lived in a 30-run test, not in a 3-million-line one.
6. Log the failures. Dozens of our PERF.md entries are reverted or not-taken experiments, and they taught us more than the wins.

The code, every PERF.md row with its board run ID, and the reverted experiments are in the repo: https://github.com/ttimesai-star/speed-demon. Thanks to Hackathon Raptors for a board that told the truth, even when we didn't want to hear it.