Independent systems + ML research · Apple Silicon

Running LLMs larger than memory
on a consumer Mac

Can an existing pretrained model be transformed post-training into an execution representation whose instantaneous working set is dramatically smaller than the full checkpoint — while preserving most of its capabilities? This platform exposes the complete paper trail: hypotheses, notes, code, benchmarks, and raw results.

total model size ≠ resident size ≠ bytes read per token ≠ parameters required for this token

304sources reviewed
24research gaps identified
6/8experiments completed
20commits

First result — the SSD substrate is not the bottleneck

Experiment H measured this Mac's internal NVMe under genuinely cold-cache conditions. Random reads reach the ~13.1 GB/s device ceiling at 1 MiB blocks (QD 8), while 4 KiB single-threaded reads manage only 67 MB/s — a 200× spread that dictates the weight-block layout contract: ≥ 256 KiB blocks at QD ≥ 4. Saturated Metal GPU compute costs < 5%.

100 MB/s1 GB/s10 GB/s4 KiB16 KiB64 KiB256 KiB1 MiBQD 1QD 4QD 8 read block size (random, F_NOCACHE cold, log–log)
Measured on this project's M5 Max (expH, run 20260812T034359Z, 3 repeats/cell, iostat-validated ceiling ≈ 13.1 GB/s). Lines: reader queue depth.
Experiment H: hypothesis, method, analysis →

Research phases

  • Phase 1 — Literature sweep (§4.1–4.10)5 theme notes · 304 sources
    done
  • Phase 2 — State-of-the-art maptechnique taxonomy + overlap analysis
    done
  • Phase 3 — Research gaps24 falsifiable approaches (G01–G24)
    done
  • Phase 4 — Candidate ranking10-axis scoring, 3–5 candidates
    done
  • Phases 5–6 — Framework & micro-experiments6/8 experiments completed
    in progress
  • Phases 7–11 — Prototypes → novelty checkevidence-driven
    pending

Latest from the research log

Full research log →

expA + expB complete: NEGATIVE — dynamic sparsity paging dead on dense SwiGLU; evidence selects the C1→C2 merge

  • Question: Is per-token FFN importance concentrated at pageable block granularity (expA), and is the important set temporally/domain reusable (expB)?
  • Experiment: One instrumented trace (SwiGLU intermediate energy, 16-neuron blocks × 28 layers × 6 144 positions) feeding both analyses. Results: results/expA_weight_concentration/20260812T044829Z/, results/expB_token_stability/20260812T045224Z/.

expF complete: NEGATIVE — layer-restricted escalation gives no leverage

  • Question: Is quantization damage concentrated in a subset of layers, so escalation can be layer-restricted and bytes-per-escalation cut proportionally?
  • Experiment: expF — 7 depth-groups × degrade-one/repair-one + repair-top-k, 4-bit affine vs bf16, Qwen3-1.7B, 48 trajectories, 17 configs. Results: results/expF_error_accumulation/20260812T043945Z/.
  • Result: REFUTED. Sensitivity spread 1.5× (kill: <2×). Top-29% of layers repair 24% of

expD complete: progressive residual coding validated; C1 operating point exists

  • Question: How fast do decisions/hidden states converge with residual stages, and does a margin-gated two-tier policy work? (gates G01 ResidualPager + C1 arithmetic)
  • Experiment: expD — affine g64 residual ladders (3/3+3/3+3+3, 4/4+4 bits) on Qwen3-1.7B, 48 trajectories, hidden states at 4 depths, two-tier policy grid. Results: results/expD_progressive_reconstruction/20260812T043508Z/.
  • Result: Monotone, fast convergence (hidden err ÷5 per stage; KL ÷30–60). 3+3 bits: 95.7%