Durability: WAL, mmap, and Copy-on-Write Snapshots
Part II — The Engine Room · seven builds, seven receipts
These experiments are receipt-class durability: you should finish each one with a hex dump, a CRC, a log line, or a passing regression test that survives a process death. The navigator's discipline applies — write the line first, then measure.
Run it live — GQL console
Point at the public read-only instance (no key needed) or your own engine
(start it with GIGI_CORS_ORIGIN=* for browser access — dev only).
—
E5.1Reproduce the 65-byte CHEMBL25 frame
Build
A twenty-line Rust harness that opens an Engine on a temp directory, creates a chembl bundle with base chembl_id and fiber pki, inserts (CHEMBL25, 9.2) through engine.insert(), drops the engine, then reads gigi.wal back raw and hex-dumps it.
Receipt
A 151-byte file with two frames whose lengths match \(4{+}78{+}4 = 86\) and \(4{+}57{+}4 = 65\); the second frame's bytes match BLD-DUR-WORKED-INSERT-FRAME from harvest/book_ch5_worked_example.rs byte-for-byte, including the six 66 bytes that are 9.2 repeating in base two.
Bonus
Swap the pKi for \(\pi\) and verify the IEEE 754 bytes match 0x400921FB54442D18.
E5.2Refute the side door, then brick it
Build
Add a test under tests/ that calls BundleStore::batch_insert directly (the bug-one path), drops the process, reopens with Engine::open, and asserts the records are gone — then a second test asserting that the same records routed through engine.batch_insert() survive.
Receipt
Two passing tests whose diff makes BLD-DUR-BUG1-IMPORT executable: one named raw_store_loses_data_on_restart (post-condition: total_records() == 0), one named engine_route_survives_restart.
Bonus
Grep src/bin/gigi_stream.rs for any remaining call site that takes a &mut BundleStore from outside the engine module; if you find one, file the receipt as a doc PR.
E5.3Instrument the streaming replay
Build
Generate a synthetic WAL of 1M, 5M, and 27M OP_INSERT entries against a fresh store, then boot Engine::open under /usr/bin/time -v (or psutil on Windows). Plot peak RSS against entry count for both the streaming path and the legacy read_all path.
Receipt
Two curves on the same axes: streaming replay holds roughly constant RSS independent of entry count (the BLD-DUR-BUG2-OOM fix); read_all grows linearly with a slope that extrapolates to the 27M “twice the dataset” peak the spec records.
Bonus
Inject a torn tail at byte \(N - 7\) in the 5M log and confirm the streaming path stops at a named offset rather than after RAM runs out.
E5.4Forge a corrupt frame and watch recovery refuse it
Build
Take the gigi.wal from E5.1, flip one bit in the middle of entry two's payload (anywhere inside the CHEMBL25 text bytes), and re-open the engine. Then repeat with a bit flip in the length prefix instead of the payload.
Receipt
The payload flip produces a hard error naming the file, the offset, and both CRCs in hexadecimal — because the length prefix rides outside the CRC, the length flip produces the distinct “ran out of file” or oversized-frame error path. Both behaviors are pinned by src/recovery.rs; your harness should print which path fired.
Bonus
Flip the CRC bytes themselves and confirm the error message still distinguishes “corrupted” from “torn” — the two cases the navigator's morning-after forensics rely on.
E5.5Re-derive the #106 union formula by counterexample
Build
Build a property test under src/mmap_bundle.rs that generates random combinations of base size, overlay size, tombstone count, and shadow count, computes len() both ways — the wrong way (base.len() + overlay.len()) and the shipped way (\(|\mathrm{base}| - |\mathrm{base} \cap (T \cup O)| + |\mathrm{overlay}|\)) — and asserts they disagree iff any overlay key shadows a base key.
Receipt
The test fails on the naive formula and passes on the shipped one; the failing seed exhibits the regression test's epigraph (“still 4 logical records, base[1] is shadowed, not added”). Cross-check against the existing test at src/mmap_bundle.rs:2454.
Bonus
Add a case where the same key is both tombstoned and shadowed and confirm the first-fix arithmetic (the one that over-subtracted by intersecting twice) gets it wrong by exactly one.
E5.6Benchmark the atomic rename window
Build
Wrap the snapshot writer in a harness that triggers a snapshot, then kill -9's the process at three points: before the .dhoom.tmp write, mid-write, and between the write and the rename. Restart and observe which chart the engine loads.
Receipt
In all three crash points, Engine::open loads the previous .dhoom (never a half-written one) and replays the WAL from its CHECKPOINT — the chart room never holds a partially drawn map. Receipt is the startup log line “WAL replay complete: N entries” where N is exactly the entry count since the previous snapshot.
Bonus
Verify the .dhoom.tmp file is removed on the next successful snapshot, not left to accumulate.
E5.7Time the overlay promotion
Build
Construct an OverlayBundle with \(N \in \{1\mathrm{k}, 10\mathrm{k}, 100\mathrm{k}\}\) records (mix of mmap base, overlay writes, and tombstones), then call to_temp_heap_store() in a loop and record p50/p99 latency. Compare against the doc comment's claim of \(\sim\)10ms for Marcella's 10k bundle.
Receipt
A table with three rows showing O(N) walk-and-insert behavior; the 10k row should land within a small constant of 10ms on commodity hardware. The promotion path is the literal bridge between BLD-DUR-COMMIT-BRAIN-POLY and the brain endpoints at src/bin/gigi_stream.rs:14792.
Bonus
Cache the result across calls and measure the savings on a workload that hits the same overlay bundle ten times — this is the optimization the doc comment leaves as future work for the caller.