Momentum — Sprint Ops Dashboard

Mission: everything local → R2; build only what changed; ship the flagship v2 rollout without breaking Netlify. Deploy doctrine (new): ship-fast.sh is the default — incremental, ~10-15 min, skips the ~30 sitewide aggregate rebuilds; run full ship.sh only when the aggregate layer changed or at the 7-day mark. R2 doctrine: copy-only, delete local only after rclone-check + explicit word.

Snapshot: Mon Jul 13, 2026 · 7:41 PM EDT — service worker live at v1076; full ship completed 4:51 PM EDT today. Heads-up: a full ship.sh looks to be running right now (ship.log writing this minute). Data archive to R2 still last verified ~Jul 10-11 — a re-sync is the top NOW item. Static numbers; ask Claude to refresh.

🚀

Today — deploys go incremental by default + the map + the popup, all live

New standing rule (in CLAUDE.md): default to ship-fast.sh for any listing/community/map-pin/report/single-script change — stop re-shipping the whole site for small edits. ship-fast now also runs the deploy-time stamp step and bumps the service-worker CACHE_VERSION, so fast deploys can no longer serve stale JS. 6 files committed. Two live bugs closed with it: the Stellar map went blank (a partial compile had overwritten stellar-pins.json down to 5 pins — now guarded, plus new build/rebuild_pins.py regenerates a feed's full pins from shards with no 75k-page re-render), and the lead popup finally updated — the cache-first SW had been pinning the old email-only version; the bump fixes it and the popup now delivers the monthly Market Pulse report.

✅

Search Console + popup — shipped, not just staged

The Dataset "missing description" error is clean in the build (0 of 1,045 data pages; the flagged May-2026 page is now an Article). The Breadcrumb "missing name" live bug (blank-address IDX records emitting an empty crumb) is patched in both _listing_render.py + _listing_render_v2.py, and today's full ship recompiled listing pages carry it. The lead popup's phone field + TCPA consent are now serving publicly after the CACHE_VERSION bump. Remaining human step: hit Validate Fix on both GSC errors so Google recrawls.

In flight right now

Running

Full ship.sh appears mid-run

ship.log is being written as of 7:41 PM EDT (Stellar compliance sweep in the tail). Last completed full ship was 4:51 PM EDT. Confirm on the Mac; let it finish before starting another deploy.

Overdue

Data archive → R2 not verified since ~Jul 10-11

Newest shard write is Jul 12 20:48; no new pulls today. Re-run _r2_data_sync.sh + rclone check --checksum before pruning any local raw.

Done today

Momentum pending bucket now compiles

_compile_momentum.py ignored the pending bucket — 79 own under-contract listings got no page/pin. Now compiles + pins active + pending (deduped).

Pre-rollout fixes — shipped today

Live in generators + shipped

Breadcrumb "missing name" (GSC)

Blank addr → empty crumb name on IDX pages. Fallback city/ZIP → community, patched in both renderers; today's full ship recompiled listing pages. Then hit Validate Fix.

Already clean

Dataset "missing description" (GSC)

0 / 1,045 data-page Datasets missing a description. Redeployed → hit Validate Fix in Search Console.

Live

Lead popup — phone + TCPA consent

Root cause was the cache-first SW, not missing code. The CACHE_VERSION bump (now v1076) + 5-min revalidating header on /lead-magnet.js mean the phone + "may call, text & email… reply STOP" version now serves. Popup also delivers the monthly Market Pulse.

Live in generators + shipped

Title / H1 leading comma

Same blank-addr root cause rendered , Pembroke Pines, FL 33025. Drops the comma when there's no street address. Rode the recompile.

Everything → R2

Done + reclaimed

Stellar photos → momentum-images

4,173,851 verified on R2 (0 differences). Local img/mls/stellar deleted → ~360 GB reclaimed.

Re-run needed

Data archive → momentum-data

Last verified ~Jul 10-11. Newest local shard write is Jul 12 20:48 (no pulls today), so it's ~1 day of deltas behind. Run _r2_data_sync.sh then rclone check; wc -l of the missing-list = the 0-diff proof.

Paused — resume

Miami media (~2.7M photos)

R2-direct. Resume anytime, runs ~6 days in background. Nothing depends on it.

Paused — resume

Spark mirror (~1.1M realMLS)

R2-direct. Resume anytime (~few days).

Done

Miami actives records

111,488 records — on R2.

Key numbers

v1076

live service-worker cache (full ship 4:51 PM EDT)

26 GB

.cro-backups — safe to reclaim now (grew from 14 GB)

1.2 TB

disk free — 36% used (last verified Jul 12)

~Jul 11

last data-archive R2 sync — re-run due

4,173,851

Stellar photos verified on R2

0 / 1,045

Dataset pages missing a description

Needs a human

NOW
Let the in-flight ship finish, then hit Validate Fix on both GSC errors (Dataset + Breadcrumb) so Google recrawls the fixed listing pages.
NOW
Re-verify the data archive: bash _r2_data_sync.sh → rclone check data r2:momentum-data/data-archive/data --one-way --fast-list --exclude "__pycache__/**" --missing-on-dst /tmp/r2-data-missing.txt → wc -l /tmp/r2-data-missing.txt. 0 = archive verified. Paste the output to Claude.
NOW
Reclaim (safe, no upload): flush.log already gone; rm -rf .cro-backups now recovers 26 GB. Do not prune the ~37 GB local data/mls/raw yet — the sync + rclone check --checksum above must pass first.
NEXT
Decide the storage architecture — one question: do Spark / MLS GRID / Trestle terms allow pulling the feed from a cloud IP? Yes → stand up the ~$9/mo Hetzner cloud ingestion worker (permanent fix for disk pressure + pull-vs-dev contention). No → on-prem box + faster uplink. Full plan in MWM-Infrastructure-Plan.md.
NEXT
Start the flagship v2 rollout in tranches (realMLS first, Glen Kernan master, 8,845-page universe) — now on the ship-fast.sh default, deploy-delta per tranche.
OPEN
Resume media + Spark (background R2-direct) whenever; nothing blocks on them.
OPEN
Decide: green-light the Cloudflare-from-R2 Phase-1 subdomain proof (CLOUDFLARE-MIGRATION-PLAN.md) — the permanent deploy fix at 10x.

Flagship rollout — realMLS first

Community / neighborhood pages

Status: v2 path confirmed schema-clean (_build_v2_nbhd.py — breadcrumb names + Dataset descriptions present). 8,845-page universe.
Method: Tranches → deploy delta → repeat. Data-backed communities first; thin ones via _v2_snapshot.py --all-slugs --resume.
Gate: Jon's phone check of the pilots before wide tranche.

Listing / individual pages

Status: Renderers GSC-patched + shipped (breadcrumb + title/H1 live via today's full ship). Encode v2 into _listing_render.py, then tranche.
Rule: ship-fast.sh per tranche, deploy-delta gate, never one big-bang deploy.

Plans & tools banked (in repo + 00-INDEX)

MWM-Infrastructure-Plan.md — the long-term storage / ingestion / build architecture: 5-tier target + 4-phase rollout (reclaim → stream-and-prune → $9/mo cloud worker → Parquet/DuckDB → serve-from-R2). In repo; awaiting the cloud-IP ToU decision to begin Phase 2.
ship-fast.sh — the new default deploy: incremental Netlify upload, skips the ~30 sitewide aggregate rebuilds, runs the deploy-time stamp + bumps the SW CACHE_VERSION. Full ship.sh only for sitewide-aggregate changes or the 7-day mark.
build/rebuild_pins.py — regenerate a feed's full map pins from shards without a 75k-page re-render (the fix for the Stellar blank-map partial-compile wipe).
INCREMENTAL-BUILD-DEPLOY-PLAN.md — ship only changed pages (deterministic output, skip-unchanged compile, delta gate). Build Layers 1–3 before the big tranches.
CLOUDFLARE-MIGRATION-PLAN.md — serve straight from R2 (R2+Workers, not Pages); deploy = incremental object upload. The 10x endgame.
R2-HYDRATE-BUILD-PLAN.md — local data/ as a disposable R2 cache (hydrate → build → check → clean).
_chunked_deploy.sh — seed Netlify's store in sub-cap draft batches, then one prod deploy (for genuine big sweeps).
hydrate.sh / clean.sh / _hydrate_lanes.json — the hydrate toolkit.
_deploy_excludes.txt — proprietary/PII data lakes blocked from the CDN (the v1069 fix).

Tomorrow + beyond

Stream-and-prune ingestion — pull writes a shard → uploads to R2 → verifies → deletes local; raw stops piling up on the Mac (no hardware)
$9/mo cloud ingestion worker (pending the ToU answer) — moves MLS pulls off the Mac; ends the pull-vs-dev disk/CPU contention
Parquet + DuckDB data layer — convert the big JSON (media-cache, pins) to columnar; generators read the working set, not 465 MB blobs
Build incremental generation (Layers 1–3) — stops every ship being a 40k-file churn; prerequisite for fast rollouts
Cloudflare Phase-1 subdomain proof — see the real deploy speed on your page set
Resume media + Spark; big-resources loop (~20M records) after
Listing-v2 encode into _listing_render.py → tranche
Optional: version-bump /lead-magnet.js?v= during a recompile as belt-and-suspenders for hard-cached browsers
Bandwidth: 125 Mbps up is fine — tame bufferbloat (router SQM); slowness is small-file + contention, not the ISP