Hypothesis — expH_ssd_feasibility#

Hypothesis
  The internal Apple NVMe SSD (AP2048Z, 2 TB) sustains enough *random* read
  bandwidth at weight-block-sized granularities (256 KB–4 MB) to stream a
  meaningful fraction of model weights per token: we expect ≥ 2 GB/s random
  reads at ≥ 1 MB blocks with F_NOCACHE (true storage path), degrading
  sharply below 64 KB, and only modest degradation under concurrent Metal
  GPU compute (unified memory contention is expected to be small relative
  to SSD ceiling).

Falsification criterion
  If uncached random reads at 1 MB blocks sustain < 500 MB/s, or collapse
  by > 50% under concurrent GPU matmul load, SSD weight streaming cannot
  supply even ~0.5 GB/token at interactive rates and candidate designs
  must not assume per-token SSD reads on this class of hardware.

Method
  Create an 8 GiB incompressible test file on the internal APFS volume.
  Measure read throughput at block sizes {4 KB, 16 KB, 64 KB, 256 KB,
  1 MB, 4 MB} × {sequential, random} × {F_NOCACHE on, off/warm cache} ×
  {1, 4, 8 reader threads}. Cold-cache condition enforced via F_NOCACHE
  (macOS has no O_DIRECT); warm condition by pre-reading the file.
  Repeat each cell 3 times, report mean/median/std. Then repeat the key
  uncached random cells while an MLX fp16 4096×4096 matmul loop saturates
  the GPU. Record hardware manifest and thermal level before/after.

Baseline
  Apple's advertised sequential throughput for this SSD class (~5-7 GB/s)
  and the sequential-read measurement from the same harness (internal
  baseline; no straw men — we compare random vs our own sequential).

Result
  Hypothesis CONFIRMED and understated. Cold random reads: 67 MB/s (4 KiB
  QD1) → 13.8 GB/s (1 MiB QD8, at the iostat-validated ~13.1 GB/s device
  ceiling). GPU load cost <5%. 4 MiB and threaded-sequential cells were
  cache-contaminated and are flagged invalid. Full numbers:
  results/expH_ssd_feasibility/20260812T034359Z/ and analysis.md.

Interpretation
  SSD streaming is viable at ~650 MB/token (realistic) if and only if
  blocks are ≥256 KiB at QD≥4. Prefetch can overlap Metal compute freely.
  See analysis.md for the full argument and the measurement-discipline
  findings (F_NOCACHE semantics).

Next experiment
  expD/expG — the substrate is not the bottleneck; whether ≤650 MB/token of
  useful bytes preserves behavior is. Secondary: rerun sequential cells with
  a 64 GiB (> RAM) file.

Analysis — expH_ssd_feasibility#

Run: results/expH_ssd_feasibility/20260812T034359Z/ · code committed at b4a652d Hardware: Apple M5 Max (6P+12E, 40 GPU cores), 48 GB unified memory, APPLE SSD AP2048Z (2 TB, internal APFS), macOS 27.0 (26A5388g), Python 3.14.4, MLX 0.32.0. 3 repeats/cell, adaptive budget (~3 s wall/repeat, 256 MiB floor, 8 GiB cap), 8 GiB incompressible test file written with F_NOCACHE. iostat -d -w 1 logged alongside as controller-level ground truth.

Hypothesis
  See hypothesis.md: ≥2 GB/s uncached random reads at ≥1 MiB blocks; sharp
  degradation below 64 KiB; modest degradation under concurrent Metal load.

Falsification criterion
  <500 MB/s uncached random @1 MiB, or >50% collapse under GPU load.
  → NOT triggered. Hypothesis survives, and understates the hardware.

Method / Baseline
  As registered in hypothesis.md (internal sequential baseline; no straw men).

Result (mean of 3 repeats; std in results.json, mostly <2%)
  Cold random reads (F_NOCACHE file, iostat-validated):
    4 KiB   QD1:     67 MB/s   (16.6k IOPS)      QD8:    480 MB/s (117k IOPS)
    64 KiB  QD1:    770 MB/s                     QD8:   5138 MB/s
    256 KiB QD1:   2302 MB/s                     QD8:  11590 MB/s
    1 MiB   QD1:   4556 MB/s   QD4: 13628 MB/s   QD8:  13789 MB/s  ← at ceiling
  iostat peak disk throughput across the whole run: 13112 MB/s → the true
  device ceiling is ~13 GB/s. Warm-cache peak (page cache / unified memory):
  136 GB/s — a ~10× RAM:SSD ratio, far better than the ~100–250:1 cliff on
  discrete-GPU systems (PCIe + separate VRAM).
  Under concurrent MLX fp16 4096² matmul load (10 032 iterations, GPU
  saturated): 4 KiB QD8 480→478 MB/s; 1 MiB QD8 13789→13393 MB/s (−2.9%).

  INVALID CELLS (flagged, not used): every cell whose mean exceeds the
  iostat ceiling is cache-contaminated — all 4 MiB cells (48–55 GB/s) and
  threaded sequential cells at ≥256 KiB (24–34 GB/s). Cause: budget ≈ file
  size, so repeats re-read blocks whose F_NOCACHE pages had not yet been
  evicted. Sequential single-thread cells ≤14 GB/s are plausible but treated
  as upper bounds only.

Interpretation
  1. SSD weight-streaming is VIABLE on this hardware — with the right access
     pattern. At ~13 GB/s, a runtime spending 50% of a 100 ms/token budget on
     I/O can read ~650 MB/token; even QD1 1 MiB reads supply ~450 MB/token.
  2. The access-pattern contract is strict: ≥256 KiB blocks at QD≥4 (or
     ≥1 MiB at any QD) reach ≥60% of ceiling; 4 KiB random is 200× slower
     than ceiling. Weight block layout must therefore bundle to ≥256 KiB —
     consistent with LLM-in-a-flash's row-column bundling rationale.
  3. Unified-memory contention is a non-issue (<5% under full GPU matmul
     load): prefetch threads can run concurrently with Metal compute.
  4. macOS measurement discipline established: F_NOCACHE does not bypass
     already-resident pages (first run read 42–115 GB/s from cache);
     test data must be written uncached; any figure above the concurrent
     iostat ceiling is contamination; 16 KiB pages make sub-16 KiB uncached
     I/O impossible in principle.
  5. Bytes/token budget implication for candidates: interactive ≥5 tok/s
     leaves ≤~1.3 GB/token of SSD reads in the theoretical limit, ≤~650
     MB/token realistically. A 100 GB-class checkpoint is usable only if
     per-token touched bytes stay ~two orders of magnitude below total size
     — which is exactly the decoupling this project investigates.

Next experiment
  expD/expG (progressive reconstruction + decision stability) become the
  binding questions: the storage substrate is not the bottleneck; the open
  question is whether ≤650 MB/token of *useful* residual/expert/sparse bytes
  suffice to preserve model behavior. Also rerun key cells with a 64 GiB
  test file (> RAM) to close the cache-contamination hole for sequential
  cells before publishing any sequential numbers.

expH_ssd_feasibility#

SSD feasibility on macOS/APFS: measured random/sequential reads, F_NOCACHE, concurrent Metal compute

Status: scaffolded 2026-08-11, not yet run.