Candidate Ranking: Phase 4 Deliverable#

Ranks all 24 gaps from research/research_gaps.md on the ten charter §7 axes, scores 1–10: N Novelty · P Technical plausibility · Mem memory reduction · BW bandwidth reduction · Q quality retention · IC implementation complexity (10 = simplest) · HW Apple Silicon practicality (Metal/MLX kernel feasibility, unified-memory fit) · PT post-training compatibility · Sig research significance · Risk (10 = lowest). Totals are unweighted sums for orientation only; selection (§3) weights significance, cluster synergy, and the measured substrate. Contrarian gaps G22–G24 are scored as experiments (cheap falsifiers): their Mem/BW are honest 1s; their value shows in Sig/IC/Risk. Binding expH facts used throughout: ≤650 MB/token SSD budget at 5 tok/s (≤1.3 GB/token theoretical), the ≥256 KiB @ QD≥4–8, 16 KB-aligned fetch contract (4 KiB QD1 = 67 MB/s, a 200× cliff), the shift/mask-cheap decode rule (i-quant LUT penalty), free compute/I-O overlap (<5% contention), and the ~35:1 RAM:SSD ratio.

1. Scorecards#

G01 — ResidualPager (total 69)#

N 8 — Paging residual planes from NVMe is SOTA §7.3.1's verified-unoccupied intersection on any OS; the encodings themselves (Any-Precision, BitStack, RRQ) are prior art, which caps this below 9.
P 6 — Successive-refinement theory and BitDelta's ~1-bit corrections are solid ground, but the load-bearing assumption — refined-set temporal stability — is unmeasured (expB), and BitStack itself warns that reconstruction overhead slows inference even fully resident.
Mem 9 — A 2–3-bit base of a 70B model is 17.5–26 GB, resident on the 48 GB target; RAM bounds base size, not model size — the strongest direct residency decoupling of any non-contrarian gap.
BW 7 — Bytes/token = base RAM reads + fetched residuals; good only if the refined set is stable and reused across tokens (the 1–2% non-overlapped ceiling), with no cross-token amortization of the kind G17 gets.
Q 7 — Full quality is recoverable in principle (planes sum to the original), but the practical operating point is whatever residual budget fits under 650 MB/token — quality is hostage to expD's recovery curve.
IC 4 — Needs a progressive encoder, block index, demand pager, and refinement application path — substantial new machinery before a single kernel is written.
HW 6 — Zero-copy unified memory and free prefetch overlap are genuine Apple-specific advantages, but bitplane decode is gated by G05; the affine-delta fallback is materially weaker.
PT 9 — Pure PTQ plus calibration; no retraining anywhere.
Sig 9 — Directly instantiates the charter §2 decoupling and criteria B/C/E; if it works it is the project's canonical representation.
Risk 4 — Dies cleanly if expD+expB show >20% of residual bytes touched per token with <50% temporal reuse; the 35:1 ratio gives little forgiveness.

G02 — MarginGate (total 77)#

N 9 — SOTA §7.3.2's narrow-but-real gap after excluding QSpec/BiLD/CALM near-misses: nobody gates storage-tier residual fetch on decision stability with a correctness story, and no paper in §5 or §6 optimizes or even reports bytes/token.
P 7 — Margins demonstrably carry signal (the entire early-exit line; QuantSpec >90% acceptance; ρ=0.82 learned error features), but the confidently-wrong quadrant (G23) is exactly the unmeasured joint distribution that could void it.
Mem 8 — Inherits G01's resident low-bit base and adds only a probe; slightly below G01 because it does not itself define the representation.
BW 9 — Bytes/token collapses to base-read + (escalation rate × residual fetch): at the measured 5–15% knife-edge prior, the largest expected per-token byte cut of any per-token scheme in the document.
Q 7 — CALM-style calibrated risk or exact rejection variants exist and are proven, but the lossy-gated form ships precisely the errors that slip the gate — quality rides on the AUROC.
IC 5 — The gate itself is trivial; calibration machinery, rollback, and integration with a pager that must exist first make it moderate.
HW 8 — Prototype needs only MLX quantized casts, logging, and control flow — no new kernels before the pager stage.
PT 9 — Calibration-only; a small learned probe is explicitly charter-permitted (§14).
Sig 10 — If the gate works, the optimization objective of local inference changes from resident-size to decision-gated bytes/token — this is the thesis of the project stated as a mechanism.
Risk 5 — expG is cheap and decisive before any engineering; the failure mode is well-defined (AUROC <0.8 or escalation >40% on any major domain) rather than diffuse.

G03 — DeltaKernel (total 67)#

N 8 — ΔY = (ΔW)X against cached activations is an unpublished primitive (SOTA §7.3.3): GMRES-IR transposed to transformer inference, never written down.
P 7 — Exact for linear ops by linearity; the two known threats (attention nonlinearity, Metal gather cost) are bounded and enumerable rather than open-ended.
Mem 4 — Saves no residency itself; the activation cache is tens of MB — this is an enabling multiplier, not a reduction mechanism.
BW 7 — Makes refinement cost proportional to residual size instead of layer size; without it every escalation or verification doubles compute and halves the whole family's economics.
Q 9 — Exact per linear layer by construction; downstream nonlinearity handling is a mapped recompute (expF), not an approximation of unknown size.
IC 5 — One fused Metal kernel plus activation-caching hooks — contained, but real kernel work.
HW 5 — Block-sparse gather matvec is exactly the access pattern Apple GPUs punish (Endor / i-quant lesson transposed); may need the G20 CPU escape hatch to survive.
PT 10 — A pure runtime primitive; the model is untouched.
Sig 7 — An enabling primitive, not a regime; its significance is inherited from G02/G17.
Risk 5 — Kill is crisp and cheap (expE_delta: ≥2× over recompute at 10% density, ≤1e-3 relative error), and failure has a named fallback (recompute + G20).

G04 — RDLayout (total 72)#

N 6 — EBCOT-style Lagrangian truncation is 30-year-old machinery and BitStack is adjacent; the novelty is compiling it against logit-KL distortion and the measured SSD contract, which is real but incremental.
P 8 — Sensitivity-weighted allocation beats uniform everywhere it has been tried (SqueezeLLM, imatrix, LASER's improve-when-truncated matrices); the optimizer is mature and the charter's compile stage budgets unlimited offline time.
Mem 7 — A static-but-nonuniform operating point strictly better than uniform bpw at matched bytes, and it is the substrate every dynamic gap pages from.
BW 6 — Compile-time only: bytes at matched quality improve, but every loaded byte is still read each token unless a dynamic gap sits on top.
Q 8 — Better-than-uniform at matched bytes is the design objective, and per-block KL attribution errors degrade the gain, not correctness.
IC 6 — An offline compiler with no runtime component; the work is bounded and parallelizable.
HW 8 — The output is the 16 KB-aligned, ≥256 KiB-bundled layout the contract demands; the decode path can stay affine (shift/mask-cheap) if G05 fails.
PT 10 — Calibration-set only.
Sig 6 — Real but possibly incremental: if imatrix-style mixing already captures most of the nonuniform gain, this is a <15%-relative-KL footnote.
Risk 7 — Worst case is a null result at modest cost; nothing downstream breaks since pagers fall back to existing formats.

G05 — MetalTrellis (total 63)#

N 7 — Zero Metal implementations of bitplane, MSB-sliced, or trellis decode exist (SOTA §7.3.5); the formats themselves are published CUDA work.
P 6 — Shift/mask-cheap by construction (passes the decode rule on paper), but bit-transpose gather coalescing and trellis sequentiality vs SIMD-group width are genuine unknowns — which is why it is an experiment.
Mem 6 — Unlocks the 2–2.5 bpw frontier on Metal (~8× vs FP16) as a component of G01/G04, never alone.
BW 7 — QTIP proves >80% of peak bandwidth is reachable with ~2 instr/weight on CUDA; success here means byte savings finally translate to tokens/s on Apple GPUs.
Q 7 — Format-neutral: quality equals the underlying encodings' published quality iff the kernel runs at speed.
IC 4 — Three hand-written Metal matvecs with correctness and throughput accounting — the most kernel-intensive micro in the plan.
HW 5 — HW practicality is the axis under test; the i-quant precedent is explicitly unfavorable and the honest prior is uncertain.
PT 10 — Kernels only; nothing touches the model.
Sig 7 — A fan-out dependency: gates whether G01/G04 use bitplanes or degrade to affine-delta residuals.
Risk 4 — Binary outcome; a loss permanently narrows the representation design space (though the affine fallback exists).

B. Sparsity as a paging policy#

G06 — TealPager (total 51)#

N 6 — TEAL/CATS/GRIFFIN exist with published numbers; the paging framing and the bytes/token measurement are new, the mechanism is not.
P 6 — Thresholds are calibration-only and GRIFFIN's per-sequence granularity is inherently prefetch-friendly, but SwiGLU mask working-set statistics are completely unmeasured (all published stats are ReLU-era).
Mem 5 — 40–50% model-wide sparsity is ~2× — rescues a 20–30% RAM deficit, nothing more.
BW 5 — The 2× FFN cut erodes further if masks churn: Jaccard(t,t+1) <0.6 makes paging traffic exceed the dense-read baseline.
Q 4 — Sirius showed contextual sparsity specifically damages reasoning/GSM8K; the 99%-retention claims are perplexity mirages, and this gap starts from that liability.
IC 4 — Metal gather kernel + threshold calibration + paging integration; three subsystems for a 2× ceiling.
HW 4 — A 50%-sparse gather matvec on a GPU with no sparse tensor cores may run slower than dense — a kill condition entirely independent of quality.
PT 8 — Calibration thresholds only, but applicability is limited to SwiGLU dense models with enough safe sparsity.
Sig 6 — The only training-free selectivity mechanism for dense stock models — important if true, but with a modest ceiling even then.
Risk 3 — Two independent cliffs (reasoning quality; gather kernel), either one fatal.

G07 — WandaTiers (total 68)#

N 7 — Importance-scores-as-reversible-residency with dense correction is SOTA §7.3.6, unoccupied; every ingredient (Wanda, Endor format, Sirius loop) is published separately.
P 7 — Demoted weights are by construction those whose absence changes outputs least, and Sirius bounds the correction bill at ~11% of tokens.
Mem 6 — 20–50% demotion runs 1.3–1.5× over-RAM dense models; not a 10× mechanism, but the cheapest real one.
BW 6 — Bytes/token = cold-block demand, favorable only if demand is rare rather than diffuse; diffuse demand lands in the 67 MB/s regime and kills it.
Q 6 — Global-average scores can miss load-bearing weights ("Accuracy is Not All You Need": 25%+ flips for pruning-type interventions); the correction loop mitigates but does not erase this.
IC 7 — The cheapest oversubscription route in the document: no re-encoding, no predictors, no new format — placement plus a dense-readable bitmap.
HW 8 — Endor-style dense-readable demoted blocks avoid the gather penalty; serve-miss-as-zero is trivial on unified memory.
PT 10 — One-shot post-training statistics; fully reversible.
Sig 6 — A working version is a solid systems result and a useful floor, not a new regime.
Risk 5 — Single crisp kill (cold-block demand >5% of demoted bytes/token at 35% demotion), measurable in expA-blocked before any runtime exists.

C. Cache, replacement, and residency policy#

G08 — MRURing (total 66)#

N 6 — MRU-for-cyclic-scans is DBMIN 1985; never applied to the dense-decode weight scan, so the novelty is purely in the (real) application.
P 8 — The access pattern is fully known (static layer order), the math is 40 years old, and the 0.025 tok/s MLX-mmap disaster is the measured counterfactual it fixes.
Mem 4 — Rescues only slightly-over-budget models: the re-read floor caps useful overflow at ~650 MB (5 tok/s) to ~6.5 GB (0.5 tok/s).
BW 6 — Achieves the theoretical floor — re-read exactly the overflow X GB/token — where kernel LRU re-reads everything.
Q 10 — Bit-exact; a pure replacement-policy change.
IC 5 — The simulator is trivial; the real ring needs explicitly managed wired buffers under the iogpu limit, i.e., engineering against the kernel-panic cliff.
HW 6 — macOS gives no UBC eviction control, forcing the wired-arena route; expH confirms the re-read stream itself runs at ~13 GB/s with free overlap.
PT 10 — No model change of any kind.
Sig 5 — Narrow applicability caps significance, but it is also the enabling floor under G18's sweeps.
Risk 6 — The simulator answers most of the question for free; kill is a measured <1.5× gain over kernel-LRU mmap at 10–20% overflow.

G09 — ArcExperts (total 75)#

N 5 — Every ingredient is published or an open llama.cpp feature request (#20757); composition novelty only, and upstream may merge layout+QD first.
P 9 — 2.23× from layout alone is already measured on a Mac (#18758), MoE temporal locality is confirmed, and ARC's superiority on mixed workloads is textbook storage systems.
Mem 6 — Runs Qwen3-30B/80B-class MoE on 16–48 GB Macs — capacity via the selectivity the router provides for free.
BW 7 — Reads/token already shown to drop 1418→370 from layout; ARC + QD≥8 explicit reads + purgeable-warm demotion stack multiplicatively on that.
Q 10 — Exact: misses block briefly, nothing is approximated.
IC 6 — Cache + bundle compiler + explicit reads inside an existing runtime; zero new math.
HW 9 — Expert bundles (MBs) natively satisfy the ≥256 KiB QD≥8 contract; the zero-copy Metal path ships today.
PT 10 — Stock checkpoints, stock router.
Sig 5 — High deployment value with a real "engineering, not research" risk if policy headroom over LRU is <5 hit-rate points at every cache size.
Risk 8 — Worst case still yields a useful runtime plus a publishable routing-trace study.

G10 — PurgeableWarm (total 57)#

N 5 — Purgeable memory is a shipped, decade-old macOS/iOS API; pointing it at weights is new but small.
P 6 — The semantics fit (a purge is exactly a cache miss), but per-access volatile-state transitions may be unaffordable and the OS purge order is model-blind.
Mem 2 — Robustness and coexistence with user apps, not reduction.
BW 2 — None directly.
Q 9 — Purge → miss → refetch at 13 GB/s; quality untouched, latency jitter only.
IC 6 — An MTLHeap wrapper plus pressure testing; contained.
HW 7 — A macOS-unique mechanism — one of the few gaps discrete-GPU prior work literally cannot express.
PT 10 — Runtime only.
Sig 4 — Valuable hygiene for every caching gap (converts panic risk into graceful degradation), not a result by itself.
Risk 6 — expJ is a cheap probe; failure just means fixed self-managed budgets.

G11 — NeverBlock (total 55)#

N 8 — Anti-caching's "never stall on a miss, repair later" has never been formulated as an architectural invariant for neural weights.
P 5 — QEP's near-exponential error compounding says a layer-10 miss contaminates the whole downstream token; repair-later may collapse into deferred blocking with extra bookkeeping.
Mem 3 — No direct reduction; it buys decoupling of token cadence from storage latency.
BW 4 — Converts I/O jitter into bounded quality jitter; average bytes unchanged.
Q 5 — Real, bounded quality noise under miss bursts, entirely dependent on expF's layer-tolerance map.
IC 4 — Correct deferred-repair bookkeeping across layers and in-flight tokens is subtle, and it presupposes a G01-style representation for graceful miss values.
HW 7 — Unified memory makes the substitute value (resident base plane) free to reach; the repair path is G03's kernel.
PT 9 — A runtime invariant over an already-permitted representation.
Sig 7 — P99, not mean, is the real interactive constraint (PowerInfer-2: +40.9% P99); a definitive answer matters for any shipped runtime.
Risk 3 — Plausibly self-killing via error compounding; mercifully, the expF injection test is cheap.

D. Prefetch and prediction#

G12 — HiddenPrefetch (total 57)#

N 6 — Eliseev–Mazur lookahead and ShadowLLM early probes exist; extending to N-layer-ahead SSD prefetch with wasted-byte penalties is incremental-but-real (Pythia's objective, never for weights).
P 6 — One early probe demonstrably shadows all layers, but the blocks that matter for prefetch (non-resident ones) are by definition the distribution's tail, where prediction is hardest.
Mem 2 — None; it spends RAM on apparatus (PowerInfer-2's predictors cost 2.6 GB at 47B — the cautionary number).
BW 6 — Does not reduce bytes; it hides latency — which expH says is fully hideable given ~1–5 ms of lead time.
Q 8 — Mispredictions cost bandwidth, never correctness.
IC 4 — Trace pipeline + probe training + integration into a pager that does not exist yet.
HW 7 — Logistic probes are microseconds on AMX and overlap is measured-free; footprint is the only HW concern.
PT 8 — Small trained probes are charter-permitted but do require a training loop and per-model calibration.
Sig 6 — The enabler that determines whether G01/G06/G07 meet their overlap assumptions; inherits significance from them.
Risk 4 — Recall <70% at ≤2× overfetch on the non-resident subset is a live possibility, and the probe+metadata budget cap (5% of RAM) is tight.

G13 — DomainCache (total 66)#

N 7 — Session-granular weight working sets are unmeasured and unbuilt; adjacent evidence (domain-correlated experts, LoRA subspaces) is suggestive, not demonstrative.
P 5 — The rival hypothesis — token-level syntactic dominance with a thin idiosyncratic tail — is equally plausible a priori; expC exists precisely because nobody knows.
Mem 6 — A domain profile bounds the resident set for an entire session if the structure exists.
BW 8 — Session-granularity amortizes fetches over thousands of tokens, making the 35:1 ratio irrelevant — the most favorable bytes/token geometry in the document, conditional on locality being real.
Q 7 — Cache warming is quality-neutral; quality is at stake only in exclusionary (G07-style) use.
IC 6 — Offline profiles + session-start warming (seconds of sequential 13 GB/s reads); light runtime.
HW 8 — Sequential warm reads are the SSD's best case, and it composes directly with the G08/G09 cache machinery.
PT 9 — Profiles are offline statistics over a stock model.
Sig 7 — Would establish semantic structure in weight usage — a scientifically interesting result independent of its systems value.
Risk 3 — One unmeasured premise carries everything; a within-vs-cross-domain Jaccard delta <0.10 kills it outright.

G14 — TrunkDelta (total 56)#

N 7 — Assigning a sharing decomposition to the memory hierarchy (trunk wired, deltas streamed in layer order) is unoccupied; the decompositions themselves are published.
P 4 — Every published sharing result needs light retraining, and BitDelta's 1-bit evidence is for finetune-vs-base pairs, not layer-vs-layer within one model — the load-bearing transfer is unproven.
Mem 7 — The trunk amortizes across L layers (read L times, stored once); if deltas quantize to 1–2 bits, resident cost collapses.
BW 7 — Delta streaming is perfectly sequential in static layer order — the one access pattern the SSD serves at full 13 GB/s with zero prediction machinery.
Q 4 — Calibration-only trunk extraction plausibly degrades enough that deltas must carry most of the information, at which point nothing is saved.
IC 4 — ALS trunk fitting + delta encoder + streaming runtime; moderate-heavy across offline and online components.
HW 7 — Dense small GEMMs plus sign-mask decode are Metal-friendly and shift/mask-cheap.
PT 6 — The charter permits calibration passes, but all published quality points used 30–40M-token retraining; the honest score reflects that dependence.
Sig 7 — Layer-vs-layer delta compressibility is a real scientific question whose answer is valuable either way.
Risk 3 — The expK kill (deltas ≥3 bits/param at a 30% trunk) is quite plausible given prior art's retraining dependence.

G15 — SpectralSplit (total 63)#

N 6 — CALDERA and the SVD family are directly adjacent; tiering the two terms (resident low-rank, paged residual) is a genuinely new but small step.
P 5 — Weights are not globally low-rank (SOTA §2.8 headline); the base may be so poor that residual demand is near-universal per token.
Mem 6 — A rank-256 hot path is ~12% of dense bytes at 4096², plus whatever residual tier is resident.
BW 6 — Same demand structure as G01 but with a worse-conditioned base and double the kernel launches.
Q 5 — The split is exact by construction, but the operating curve is plausibly dominated by G01's precision split at equal bytes — the explicit kill condition.
IC 7 — Both components run on existing fast MLX kernels today; no custom Metal work gates a prototype — the cheapest Group A/E prototype to stand up.
HW 9 — Dense small GEMMs are the one decomposition Metal already runs at full speed.
PT 9 — SVD plus quantization, calibration-only.
Sig 6 — Mostly valuable as the controlled head-to-head that decides precision-split vs spectral-split for the progressive family.
Risk 4 — The kill likely triggers on the base-alone-KL arm (>1.0 nats), but the experiment is nearly free given the expD ladder.

G16 — SeedSynth (total 62)#

N 7 — SeedLM exists (Apple-authored, FPGA-validated); the Metal question and the seeds-as-nearly-free-prefetch composition are unexplored.
P 5 — LFSR generation is shift/XOR-only, but QTIP's ~2 instr/weight ceiling is demanding, and compile-time coefficient search at 70B scale is heavy.
Mem 7 — Seeds + coefficients shrink resident and stored bytes together, ~10× per block against 4-bit.
BW 9 — Attacks bytes/token below the information-theoretic floor of stored-bits formats — the strongest pure-bandwidth story in the document if the kernel wins.
Q 4 — Hard-capped at 4-bit parity with no path to exactness: it can never serve as a verification tier, only as a base.
IC 5 — One fused generate+matvec Metal kernel for the probe; the full compiler is heavy but deferrable.
HW 6 — Targets the real ALU:BW imbalance at batch-1 decode, but the i-quant precedent shows Apple GPUs punish decode work; genuinely uncertain.
PT 9 — Data-free re-encoding of a stock checkpoint.
Sig 7 — "Weights need not be stored, only recoverable" is the project's thesis in its purest form; even a negative Metal result is a useful boundary.
Risk 3 — Single-kernel kill at <80% of the 4-bit baseline, plus reproduction risk of SeedLM's parity claims on SwiGLU models.

F. Scheduling, speculation, and amortization#

G17 — BlockVerify (total 76)#

N 8 — SpecExec's economics applied to residual planes with rollback is exactly the "block-deferred verification policy" SOTA §8 lists as open; QuantSpec verifies every block from resident 16-bit weights, never from storage.
P 7 — The rejection-sampling proof carries over unchanged (same support, base=draft), and 4-bit acceptance priors are measured (>90%); acceptance at the 2–3-bit bases real oversubscription requires is the open coin-flip.
Mem 8 — Same resident low-bit base as G01; residuals never resident, read once per accepted block.
BW 9 — Divides residual bytes/token by the acceptance length, relaxing the brutal 1–2% per-token ceiling to 10–20% of weight bytes per sweep — and sweeps are sequential-by-layer, the SSD's best pattern.
Q 9 — Exact by construction under the standard rejection rule; quality risk lives only in throughput, not in outputs.
IC 5 — Drafting loop + rollback + residual-sweep orchestration; mitigated by MLX already shipping speculative verification kernels (ReDrafter) on Metal.
HW 7 — Sequential residual sweeps at QD≥8 are contract-native; the open cost is Metal batch-verification width sharing unified-memory bandwidth.
PT 9 — Base and residual encodings are PTQ; no training anywhere.
Sig 9 — If mean acceptance ≥4–7 at a 3-bit base, this makes 1.5–2× oversubscription interactive with exact outputs — criteria B and D simultaneously.
Risk 5 — Acceptance at low-bit bases is unmeasured (the G24 exposure), but the expG acceptance-length variant measures it before any engineering; rollback economics degrade it gracefully.

G18 — SpecSweep (total 78)#

N 6 — A port of SpecExec to unified memory + NVMe; the platform mechanics (zero-copy Metal buffers, no PCIe staging, 13 GB/s overflow sweeps) are new, the idea is not.
P 8 — The arithmetic uses only measured numbers (expH ceiling × SpecExec acceptance), and speculation already ships on Metal in both llama.cpp and MLX.
Mem 7 — Dense stock models beyond RAM with zero model transformation: 20 GB of overflow sweeps in ~1.5 s; ~1.4× oversubscription plausibly interactive.
BW 8 — Overflow bytes ÷ ~20 accepted tokens/sweep; the amortization factor is measured elsewhere, not hypothesized.
Q 10 — Exact rejection sampling; the target distribution is untouched by construction.
IC 7 — No new representation and no new kernels: a draft model, tree verification, and an overflow sweep (plus G08's ring for sweep stability).
HW 9 — Everything runs on shipped Metal paths today; the one unknown is RAM-vs-RAM contention between batch verification and the concurrent sweep — the contention expH did not measure.
PT 10 — Stock checkpoints, stock draft model.
Sig 6 — Doubles as the honest baseline every representation-heavy gap must beat at equal oversubscription; even its negative result is load-bearing for the project.
Risk 7 — Acceptance shortfall degrades throughput gracefully rather than killing the mechanism; the kill (accepted length <5, or <2 tok/s compound at 1.4×) still leaves a working exact runtime.

G19 — RouterBias (total 66)#

N 6 — Trained cache-conditional experts exist; the zero-shot measurement is genuinely missing but conceptually tiny.
P 6 — Near-tied gates are common under load-balancing pressure, yet the trained systems exist precisely because zero-shot biasing may not preserve quality.
Mem 4 — Raises effective cache capacity on over-budget MoE; no direct reduction.
BW 6 — A 10–15-point hit-rate lift multiplies directly into tokens/s where misses dominate token time.
Q 4 — Systematically starving rarely-resident experts risks distribution drift over long generations — a failure aggregate PPL won't show but flips will.
IC 10 — ~5 lines in mlx-lm plus a λ sweep; the cheapest intervention in the entire document.
HW 10 — Pure logit arithmetic; no kernels, no I/O, no formats.
PT 10 — A runtime knob on a stock router.
Sig 4 — A knob, not a regime: highest value-per-effort in the document, lowest ceiling.
Risk 6 — Cheap to test, but the kill (every useful λ costs >2% flips or >1 pt GSM8K) is quite plausible given why the trained variants exist.

G20 — AmxSidecar (total 61)#

N 6 — Kernel-affinity splitting on unified memory is unexplored precisely because capacity splitting is meaningless there — nobody looked; still an infrastructure idea, not a mechanism.
P 7 — P-cores idle during decode, AMX sustains ~100+ GB/s, copies are zero by construction; the unmeasured hole is RAM-bandwidth contention (expH tested SSD-vs-GPU, not CPU-vs-GPU).
Mem 2 — None.
BW 3 — No byte reduction; it reroutes irregular work (gathers, ΔY, probes) off the GPU's weakest path.
Q 9 — Numerically identical work on a different engine.
IC 6 — Accelerate GEMV + shared-event sync; the microbench is a day, production scheduling more.
HW 8 — Exploits Apple-specific silicon (AMX/Accelerate) that every CUDA-derived design structurally ignores.
PT 10 — Pure runtime.
Sig 4 — A contingency: the escape hatch for G03/G06/G15 gather-kernel failures, not a result in itself.
Risk 6 — expM is cheap; failure merely closes the escape hatch (kill: CPU <50 GB/s under GPU saturation, GPU loses >15%, or sync >50 µs/layer).

G. Attention-side working set#

G21 — KVBudget (total 69)#

N 4 — SparQ, KIVI, KVQuant are all published and KV quantization on Metal is "solved-enough"; the Metal SparQ port and joint weight+KV accounting are engineering with a thin research veneer.
P 9 — All ingredient algorithms are training-free and validated; mlx-lm's quantized-KV plumbing exists to extend.
Mem 5 — Frees gigabytes of residency at long context — a direct multiplier on every weight-side cache's budget.
BW 6 — Up to 8× attention-transfer cut where context is long; possibly <10% of per-token bytes at ≤32k — the self-mooting risk.
Q 9 — Published methods are near-lossless at 3–4-bit KV.
IC 7 — An extension of existing quantized-KV paths plus one gather kernel.
HW 8 — RAM-resident gathers only — the 35:1 SSD ratio never applies; the Metal gather cost question is the sole concern.
PT 10 — A runtime cache format.
Sig 3 — Explicitly self-demoting: expN_kvshare (one afternoon) may strike it to Phase-10 engineering; its main role is to protect other gaps' accounting.
Risk 8 — Nothing fails; the worst case is "not a priority," established almost for free.

H. Contrarian gaps (scored as experiments — cheap falsifiers, not architectures)#

G22 — WarmEnough (total 71)#

N 6 — Warm steady-state fault accounting for selective workloads on macOS is unpublished; the mechanism under test (UBC + kernel LRU) is ancient.
P 8 — llama.cpp + dtrace + wired ballast yields a decisive number in days; every dramatic prior number (0.025 tok/s, 2.23×) is cold-start or dense-cyclic, so the warm question is genuinely open.
Mem 1 — Reduces nothing; it can only delete engineering (Group C) from the plan.
BW 1 — None; it measures whether the kernel already delivers the bandwidth story.
Q 10 — A measurement; quality untouched.
IC 8 — Scripts around existing tools; no new code paths.
HW 10 — Runs on the shipped stack as-is.
PT 10 — Nothing touches any model.
Sig 8 — A <10% warm fault-time share at 75% residency demotes ~40% of the document's engineering (G08–G11); >25% validates Group C — enormous information either way.
Risk 9 — Cannot fail as an experiment; both outcomes are directly actionable.

G23 — ConfidentlyWrong (total 76)#

N 8 — P(disagree | margin) for quantized bases — the joint distribution, not the marginals — has never been published (SOTA §6.7), despite a crowded escalation literature built on its assumed shape.
P 9 — Pure logging and analysis over the expG run matrix; the measurement itself cannot fail.
Mem 1 — None; it gates the gaps that claim reductions.
BW 1 — None.
Q 10 — A measurement.
IC 9 — Shares the expG run with G02, G17, and G24 — near-zero marginal cost.
HW 10 — MLX quantized casts plus logging; no kernels.
PT 10 — Nothing modified.
Sig 9 — Single-handedly decides whether the escalation family is the project's center (AUROC >0.85) or dead (<0.65 at a 3-bit base) — the highest information-per-compute-hour in the plan; the gaps doc orders it first for this reason.
Risk 9 — Decisive either way; both outcomes are publishable guidance for the whole cheap-first field.

G24 — BasinCollapse (total 74)#

N 7 — Agreement-with-full-model and recovery-smoothness-per-residual-byte at 2–3-bit PTQ bases are unreported; the ParetoQ transition it probes is published but measured only as standalone quality, never as draft quality.
P 9 — MLX quantization at four bitrates plus reference logging; entirely mechanical on the shared ladder.
Mem 1 — None directly; it sets the resident-base floor every progressive gap must respect.
BW 1 — None.
Q 10 — A measurement.
IC 8 — The quantize/refine ladder it needs is shared infrastructure for eight other gaps.
HW 10 — Stock MLX.
PT 10 — Nothing modified.
Sig 9 — Fixes the project's oversubscription ceiling: 3-bit agreement <75% caps the whole progressive family at ~1.5–1.6× and reframes the charter §15 stretch target as unreachable by this route — the single number that stretch goal lives or dies on.
Risk 9 — Any outcome is a measured ceiling worth recording (charter §17 explicitly values this).

2. Summary table#

Gap	Name	N	P	Mem	BW	Q	IC	HW	PT	Sig	Risk	Total
G18	SpecSweep	6	8	7	8	10	7	9	10	6	7	78
G02	MarginGate	9	7	8	9	7	5	8	9	10	5	77
G17	BlockVerify	8	7	8	9	9	5	7	9	9	5	76
G23	ConfidentlyWrong	8	9	1	1	10	9	10	10	9	9	76
G09	ArcExperts	5	9	6	7	10	6	9	10	5	8	75
G24	BasinCollapse	7	9	1	1	10	8	10	10	9	9	74
G04	RDLayout	6	8	7	6	8	6	8	10	6	7	72
G22	WarmEnough	6	8	1	1	10	8	10	10	8	9	71
G01	ResidualPager	8	6	9	7	7	4	6	9	9	4	69
G21	KVBudget	4	9	5	6	9	7	8	10	3	8	69
G07	WandaTiers	7	7	6	6	6	7	8	10	6	5	68
G03	DeltaKernel	8	7	4	7	9	5	5	10	7	5	67
G08	MRURing	6	8	4	6	10	5	6	10	5	6	66
G13	DomainCache	7	5	6	8	7	6	8	9	7	3	66
G19	RouterBias	6	6	4	6	4	10	10	10	4	6	66
G05	MetalTrellis	7	6	6	7	7	4	5	10	7	4	63
G15	SpectralSplit	6	5	6	6	5	7	9	9	6	4	63
G16	SeedSynth	7	5	7	9	4	5	6	9	7	3	62
G20	AmxSidecar	6	7	2	3	9	6	8	10	4	6	61
G10	PurgeableWarm	5	6	2	2	9	6	7	10	4	6	57
G12	HiddenPrefetch	6	6	2	6	8	4	7	8	6	4	57
G14	TrunkDelta	7	4	7	7	4	4	7	6	7	3	56
G11	NeverBlock	8	5	3	4	5	4	7	9	7	3	55
G06	TealPager	6	6	5	5	4	4	4	8	6	3	51

Totals span 51–78. Unweighted totals reward safety: the falsifiers and G18 rank high partly because they cannot fail badly, while G01 (the project's canonical representation) is dragged down by IC 4 / Risk 4 — selection below therefore weights Sig, cluster synergy, and the substrate contract, not raw totals. G21 (69) is unselected as self-demoting pending expN.

3. Selected candidates for prototyping#

Four candidates: three architecture compositions (following the natural clusters named in research_gaps.md) plus one falsifier battery that gates them; all share the build-once infrastructure listed there.

Candidate 1 — Progressive-Residual Runtime (G01 + G02 + G04, with G03 kernel; gated by G05)#

Selection rationale. The composition the entire SOTA analysis points at: a 2–3-bit base resident in unified memory (17.5–26 GB for a 70B model on the 48 GB target); residual planes compiled by G04 into KL-optimal, 16 KB-aligned, ≥256 KiB embedded bundles on NVMe; fetched only when G02's margin/learned-feature gate says the token decision is not yet stable; applied via G03's ΔY kernel at cost proportional to residual size. It attacks criteria B (bytes/token ≪ compressed checkpoint: base RAM reads + 5–15% escalation × residual fetch), C (token-dependent fraction of model information), D (progressive/conditional execution), and E (a post-training representation with a qualitatively better storage/RAM/quality tradeoff than fixed quantization), and satisfies the substrate contract by construction: escalation on 5–15% of tokens fits under 650 MB/token, prefetch overlap is free, G04/G05 keep decode shift/mask-cheap. Highest-Sig cluster (10 + 9 + 6) despite mid-pack totals — exactly where totals mislead.

Promote / kill (expG, expD, expB, expF, expE variants).

PROMOTE: expG AUROC ≥0.8 for margin(+learned features) predicting base-vs-reference disagreement, with escalation ≤40% at 99% effective agreement on every major domain; expD reaching KL ≤0.05 to FP16 touching ≤20% of residual bytes/token; expB temporal reuse of the refined set ≥50%; expE_decode progressive variants ≥70% of affine-baseline throughput; expE_delta ΔY ≥2× over full recompute at 10% density.
KILL: expG AUROC <0.8 or escalation >40% on any major domain (G02 dies; fall back to Candidate 2's ungated verification); expD >20% residual bytes AND expB reuse <50% (steady-state traffic >650 MB/token — G01 dies at 70B scale); expE_decode both variants <70% (G05 dies; degrade to affine-delta residuals and re-score); G24's 3-bit agreement <75% on chat/code or non-smooth recovery (family capped at ~1.5–1.6× oversubscription — record the ceiling, pivot).

Candidate 2 — Amortized Verification Sweeps (G17 + G18, on a G08 MRU floor)#

Selection rationale. The speculation cluster converts the SSD's one great strength — sequential 13 GB/s sweeps — into exact over-budget decoding by dividing swept bytes across accepted token blocks. G18 is buildable today with zero model transformation (stock GGUF/MLX checkpoints, shipped Metal speculation): a 20 GB overflow sweeps in ~1.5 s, so ~20 accepted tokens/sweep gives an I/O ceiling near 13 tok/s at 1.4× over-RAM — criterion A (a model larger than unified memory at interactive latency) with exact outputs, plus F (speculative sweep scheduling materially changes out-of-core economics). G17 is the research upgrade: the same amortization applied to residual planes, relaxing Candidate 1's per-token traffic ceiling by the acceptance length while keeping rejection-sampling exactness. G18 is also the charter §8.4-honest baseline Candidate 1 must beat at equal oversubscription — funding it is epistemically mandatory, and its worst case is still a working exact runtime.

Promote / kill (expL_specsweep, expG acceptance variant, expI_cachesim).

PROMOTE: expL accepted length ≥5 with a 1B draft against 70B-Q4 at tree width 16–64, and Metal batch verification sustaining ≥2 tok/s compound at 1.4× over-RAM under a concurrent 13 GB/s stream; expG-variant mean accepted block length ≥4 at a 3-bit base on chat/code (G17 economics hold).
KILL: expL accepted length <5 or verification-under-streaming drops the compound ceiling below 2 tok/s (G18 demoted to baseline-only); expG-variant accepted length <4 at 3-bit base (G17 dies — sweep bytes ÷ 4 exceeds the per-token budget for 1.5× over-RAM models); expI/expO showing the swept overflow set unstable under MRU (<1.5× over kernel-LRU) removes the G08 floor.

Candidate 3 — MoE Residency Engine (G09 + G12 + G19, with G10 warm tier and G04-style bundle layout)#

Selection rationale. The cache/prefetch cluster is the nearest-term deployable win and the only candidate whose per-token selectivity is free (the router computes it): compiled 16 KB-aligned expert bundles (natively meeting the ≥256 KiB QD≥8 contract), an ARC ghost-list cache, hidden-state prefetch N layers ahead (lead time ≥ fetch latency, which expH makes sufficient), zero-shot router bias as a cheap multiplier, purgeable-warm demotion for graceful pressure behavior. Quality is exact, every ingredient has isolated measured evidence (2.23× from layout alone on a Mac), and it serves criteria A (Qwen3-30B/80B-class MoE on 16–48 GB Macs, interactive) and F (cache/prediction/scheduling changing out-of-core economics). Its known weakness is the "engineering, not research" risk — precisely what its kill thresholds test, and why it runs on shared infrastructure rather than monopolizing effort.

Promote / kill (expB MoE-trace variant, expO_warmsteady, expJ_purgeable, G19 sweep).

PROMOTE: ARC beats LRU by ≥5 hit-rate points at some cache size in trace replay (25/50/75% of expert pool); prefetch probe recall ≥70% at ≤2× overfetch for non-resident experts; router bias lifting hit rate ≥10 points at ≤2% flips and ≤1 pt GSM8K.
KILL: ARC <5 points over LRU at every cache size AND expO warm fault-time share <10% at 75% residency (policy headroom ≈ 0 and the kernel already suffices — demote to llama.cpp-upstream engineering, keep only the layout compiler); expJ transition overhead >10 µs/block/token kills the purgeable tier only; G19's kill (every useful λ costs >2% flips or >1 pt GSM8K) strikes the bias knob only.

Candidate 4 — Gate Zero: the contrarian falsifier battery (G23 + G24 + G22, plus the G21 expN probe)#

Selection rationale. Selected as experiments, per the gaps doc's own framing: the cheapest instruments in the project, and they arbitrate everything above before engineering begins. G23/G24 ride the single expG run matrix (which also serves G02/G17) and decide whether the escalation/progressive families are the project's center or dead; G22 (llama.cpp + dtrace, days) decides whether Group C custom residency is research or redundant; expN (an afternoon) decides whether KV growth silently invalidates every weight-side budget. Charter §17 makes concluding "it doesn't work" a first-class deliverable, and the fan-in analysis shows these four measurements gate the priority or design of seventeen of the twenty-four gaps. If all contrarians "win," the documented salvage architecture (G18 + G09 + G04: speculative sweeps over a well-laid-out, kernel-cached, R-D-compiled model) is itself unoccupied territory — even total premise failure yields Candidates 2 and 3 plus a publishable negative map.

Decision thresholds (these ARE the promote/kill evidence for the whole portfolio).

G23: AUROC <0.65 (margin+entropy+learned features, 3-bit base) ⇒ G02 and G17's gated forms die; block-level rejection sampling (G17/G18 ungated) survives. AUROC >0.85 ⇒ the escalation family becomes the project's center.
G24: 3-bit same-top-token <75% on chat/code, or KL plateaus until >50% of residual bytes ⇒ the progressive family is capped at ~1.5–1.6× oversubscription; record the measured ceiling; Candidate 1 is demoted and Candidate 2 promoted to primary.
G22: warm fault-time share <10% of token time at 75% residency ⇒ G08–G11 demoted to engineering polish (Candidate 3 loses its cache-policy arm, keeps layout+prefetch); >25% ⇒ Group C stands as designed.
expN: KV share <10% of per-token bytes at ≤32k for 8B–30B models ⇒ strike G21 from the candidate list to Phase-10 engineering; otherwise every candidate's RAM budget is re-derived with KV co-modeled.

4. Preferred candidate#

If one candidate defines the project, it is Candidate 1, the Progressive-Residual Runtime (G01+G02+G04+G03) — the only one that rewrites the relationship charter §20 names as the ideal outcome. Candidate 2 amortizes reads and Candidate 3 exploits existing routing, but both leave hardware coupled to model size (the sweep scales with overflow bytes; the expert pool sets tier sizes). Candidate 1 alone makes required RAM a function of the base representation (a compile-time constant chosen to fit the machine) and per-token cost a function of decision uncertainty (escalation rate × residual fetch) — additional model size lands on NVMe as embedded residual planes that cost latency only when, and in proportion to how often, the token decision actually needs them. That is instantaneous useful working set → hardware required in mechanism form, on the one platform (unified memory, zero-copy Metal buffers, 13 GB/s NVMe with free overlap) where its mechanics are measured viable, while keeping the original pretrained weights as the sole source of intelligence. It is also the portfolio's riskiest member — hence Gate Zero runs first, Candidate 2 stands as its honest baseline and fallback, and its kill thresholds are numeric and scheduled before any runtime engineering. Next (Phase 5/6): build shared infrastructure in the gaps doc's order — expG run matrix first, then the expA/B/C trace campaign, then the expD/expE ladder — before candidate-specific code.