SH3-5 round 2: Rust AST perf — remaining 10.2x hotspots and optimization plan

johnsoncodehk

## Background SH3-5 round 1 (`&'static str` typ/keys + move semantics, b82f1a9) brought the typescript-shape ast/cst ratio from **22.6x down to 10.2x** (2MB input, 32/32 gates passing). This issue records the round-2 profiling results and remaining work. ## Profiling setup - Binary: `/tmp/shape-rust-gate/profile-loop2` (typescript-shape.rs + loop main, `rustc -O`) - Input: `/tmp/shape-rust-gate/typescript-bench-2mb.ts` (2MB) - Tool: macOS `sample` for 8s, report at `/tmp/sh35-round2.sample.txt` ## Hotspots (excluding main-thread join wait and startup samples) Roughly **60% of work time is still in malloc/free** (malloc_tiny 733 + free 686 + memmove/memset samples). Sources by contribution: ### 1. `shape_restore` (384) + `drop_in_place<AstValue>` (274) On backtracking, `kids.truncate()` recursively drops already-built subtrees; and every checkpoint via `shape_ck()` `clone()`s both suppress vectors (`src/target-rust.ts` L4340–4348), with restore doing whole-Vec assignment again. **Fix (O3)**: if the suppress vectors are append-only, store only their lengths and restore via truncate, saving two Vec clones + two drops per checkpoint. Apply the same change to `shape_tpl_snap`/`shape_tpl_restore` (L4326–4338). ### 2. `AstValue::clone` (126) + `Vec::clone` (68) — fixture customs The codegen side is fully cleaned up; all remaining deep copies live in `test/fixtures/shape-typescript-rust-customs.rs` (~900 lines): - `ctx.kids.get(N).cloned()` throughout the `estree_*` functions (deep subtree copies) - `first_kid` / `flat_kids` borrow then clone - `fields.iter().find(...).map(|(_,v)| v.clone())` (ForIn/ForOf/For destructuring) **Fix**: the `ast_custom` dispatch already owns `ctx` by value; change the `estree_*` functions to take `&mut AstCustomCtx` (or by value) and extract kids via `mem::replace`/`mem::take`, driving `AstValue::clone` to zero. ### 3. Wasted construction on backtracking (architectural, deferred) `parse_ast_Decl`/`parse_ast_Stmt` self-samples are non-trivial; part of it is AST built inside failing alts then discarded by truncate. Possible direction: probe-first fast path before construction. Lower ROI than items 1–2, so deferred. ## Execution order - [ ] O3: length-based snapshots for ShapeCk/TplSnap suppress vectors (small change, targets hotspot 1) - [ ] De-clone fixture customs (larger change, targets hotspot 2) - [ ] Re-bench + re-sample, then decide whether to attack hotspot 3 ## Acceptance - `node --experimental-strip-types test/shape-rust.ts` passes 32/32 - Median ast/cst on the 2MB paired timing is clearly below 10.2x

johnsoncodehk

Round 2 done — both work items landed, acceptance met. ## Changes **O3 — snapshot de-clone (`src/target-rust.ts`)** The suppress vectors turned out to be *not* append-only (they are wholesale-replaced by suppress steps and `mem::take` at Pratt entry), so length-based snapshots were not applicable. Instead the Rust codegen now matches the TS reference exactly: TS snapshots just hold `_suppressNext`/`_suppressCur` **references** and never mutate in place, so `ShapeCk`/`ShapeTplSnap`/`ShapeParser` now use `Option<Rc<[u16]>>` — checkpoint/restore are refcount bumps instead of two Vec clones + two drops per checkpoint. **De-clone fixture customs (`test/fixtures/shape-typescript-rust-customs.rs`)** All `estree_*` functions now take `AstCustomCtx` by value and extract kids via `mem::take`/`mem::replace` (`take_kid`/`take_opt`/`take_field`/`flat_take`). `AstValue::clone` survives only where one slot genuinely feeds two output fields (ImportDeclaration arm, TSMappedType/TSIndexSignature last-kid aliasing, ≤2-kid arrow/param overlap). Behavior verified identical: old vs new binary output diff on smoke set, then full accept-corpus TS↔Rust iso **0/1403 divergences**. ## Acceptance - `node --experimental-strip-types test/shape-rust.ts` → **32/32** ✓ - 2MB paired timing: `cst≈33.5ms ast≈248ms`, **median ast/cst = 7.43x** (was 10.2x, −27%) ✓ ## Re-sample (same setup, 200 iters) Top-of-stack vs round-2 numbers: - `AstValue::clone` / `Vec::clone`: **gone** from the profile (was 126+68) - `shape_ck`: 4 samples (was part of the 384+274 hotspot) - Remaining self-time: `shape_restore` 730 (now almost entirely `kids.truncate` drop work — `drop_in_place<AstValue>` 329 under it), malloc/free xzone churn, and `parse_ast_Stmt`/`parse_ast_Decl` self samples i.e. everything left is **hotspot 3** (AST built inside failing alts then dropped on backtrack — the deferred architectural probe-first direction). Decision: acceptance is met and hotspot 3 is a larger, riskier change — deferring to a follow-up rather than bundling it here.