GitHubivarvong

ply

main
ply5 git calls93 ms39 KiB3 cachedprotocol v2
423 lines46 KiBRaw

ply — project context

This is the hand-over document: what ply is, how it is put together, how it was built, every decision that matters and the evidence behind it, how to verify all of it again, and what is still open. README.md is the user-facing guide; SAFETY.md is the invariant ledger (each claim, the test that proves it, the findings and the run record). This file is the map between them.

As of 2026-10-20: 35 modules in src/ (3,606 lines), 113 deterministic tests in 19 suites plus 16 fuzz stages in npm test (≈29 min, all green), 100% line coverage of src/ (2,461 lines, npm run coverage, reproducible seeds).


0. The name

ply — to ply is to work a route steadily, back and forth (a ferry plies the river): this client works the wire between your code and a repository it never brings home. A ply is also one thin layer of something larger — a one-line delta on a 10 GB repository. Renamed from gitwire on 2026-10-23: git's trademark policy (git-scm.com/about/ trademark, §2.3) does not allow "Git" "as a syllable in a new word or as part of a portmanteau" for a third-party product; "… for Git" is the permitted form, so the tagline is "ply — a git client for anywhere". The bare npm name ply is an abandoned 0.1.0 package: publish under a scope. Everything was renamed — package name, user agent (ply/0.1.0, agent=ply/…), request headers (x-ply-cache, x-ply-pack-cache), span attributes (ply.*), isPlyCode, environment variables (PLY_*) — so traces and server logs say ply.

1. What ply is

A zero-dependency JavaScript git client for Cloudflare Workers (and Node), speaking git's own wire protocols — smart HTTP (protocol v2 for fetch, v0 receive-pack for push) — with no clone, no working tree and no local object store.

What it does:

  • Edit one file in a huge repository in O(1) memory: ls-refs → fetch the commit (tree:0, depth 1) → one tree per path level → stream the blob through inflate, find/replace and SHA-1 without holding it → push a thin pack whose blob is a delta against the old one (hundreds of bytes for a one-line change in a multi-GB repo). Compare-and-swap on the ref: a lost race is cas_conflict, never a clobber. Optional SSH commit signing, verification read-back, LFS-pointer guard.
  • Land branches with git operations only: create (CAS), fast-forward, merge commit, delete; rebase, squash, cherry-pick, revert — all on one three-way tree merge plus a line merge ported from git's xdiff, so clean results are byte-for-byte what git merge-tree produces.
  • A general client (ls-refs, fetch/negotiate, push, discovery) for everything else, tree-shakeable so an edit-only Worker carries ~42.6 KB minified.
  • Telemetry: OTLP/JSON spans for every wire round trip and every operation, with bounded events, measured vs estimated CPU, cache status, retry links.

Design constraints, held throughout: zero runtime dependencies; Web APIs only in src/ (no node: imports — checked by the bundle test); pragmatic functional style (pure step functions and state machines, effects at the edges, free functions over classes so bundlers can drop what you do not import); every wire-sized allocation bounded by limits.js; every error typed (errors.js, one table with a retry policy).

2. Map of the code

Layers, bottom up (each depends only on the ones below):

LayerModulesNotes
Codecs (pure)bytes, pktline, inflate, sha1, objects, deltaenc, pack, packresolve, validatestreaming where it matters (inflate reports bytes consumed; SHA-1 is incremental)
Errors & limitserrors, errors-doc, limits27 codes, each with a retry policy (yes/fresh/no/read); docs kept apart so the edit bundle does not carry them
Transport contracttransport, httpa transport is a plain record { frames(service, body, span, hints), advertise(service, span) }; one implementation (HTTP), no dispatch
Protocollsrefs, fetch, fetchpack, negotiate, push, receivecaps, discover, packcachev2 fetch as a pure state machine (v2Init/v2Step); push report parsing pure
Operationsoperation, edit, tree, checkout, log, merge, linemerge, attributes, rebase, signoperation() owns the root span, transport, outcome and the "after an acknowledged push, any read-back failure is ambiguous" rule
Entry pointsedit.js (edit-only bundle), tree.js (whole-tree push), checkout.js (whole-tree read), index.js (everything), client.js (GitHttp convenience wrapper); Node-only adapters outside every Workers bundle: tree-node.js and checkout-node.js (node:fs)sideEffects: false
Telemetryotelminimal OTLP/JSON tracer; exporters pluggable (test/jsonl.mjs for files)

Key operation modules:

  • edit.js — editFile(url, spec, opts): find/replace in one path, create-on-branch (ref + base), head hint (ls-refs in parallel with a speculative commit fetch), pack cache, signing, verify, LFS guard.
  • tree.js — pushTree(url, spec, opts): a caller-given file set (object, Map or iterable of { path, mode, data|oid }, symlinks 120000, gitlinks 160000) as one commit, with editFile's head hint, fresh cas_conflict retry, strictHead, verify/ambiguous, signing and tree cache. Thin repeat pushes: the parent's trees are read only where the new tree differs, known blobs are not sent, changed trees go as REF_DELTAs. tree-node.js (filesFromDir, pushDir) is the Node directory walk. Issue #11.
  • checkout.js — checkoutTree(url, { ref, path }, opts): the mirror of pushTree. ls-refs → one blob:none fetch of the commit (every tree) → blobs in batch-sized wants lists (buffered, delta-resolved, keyed by computed hash; an unwanted object is pack_corrupt), a batch over a cap split in half down to single blobs that stream through read.js's openBlob; a host refusing many wants gets one per request (quirks.blobWants). Returns entries and a files async iterator. checkout-node.js (writeFiles, checkoutDir) writes a directory as git clone lays it out and never writes through a symlink. Issue #12.
  • log.js — readLog(url, { ref, limit }, opts): first-parent history (git rev-list --first-parent -n) from one deepen <limit> fetch through fetchPackTree0 (tree:0, else blob:none with the trees harvested). logWindow (pure) holds the window to the #37 deepen contract (the want present; every commit an ancestor; a missing parent only under a shallow line, else pack_corrupt) and sets truncated from the last commit's parents. No path filter. Issue #56.
  • merge.js — mergeBranch, fastForward, deleteRef, mergeTrees (trivial-merge rules per directory, only differing subtrees fetched), bestBases (git's merge-base --all with a completeness check), rename guards, blobMerger, attributeDrivers/attributeTree.
  • linemerge.js — port of git's xdiff as merge-ort drives it: histogram diff (+ hash-chain limit), classic Myers fallback (prepare-stage trim/cleanup, xdl_split heuristics), group compaction (+ git 2.54's shifted-group re-diff), build script, xdl_do_merge, conflict refinement and simplification, fill (clean and merge=union). Its diff work is counted (every inner-loop iteration of the histogram scan and the classic fallback, on the module's own counter) against limits.maxMergeWork, absolute per merged file: past it the file is a conflict with reason too large to merge, as one over maxMergeBlobBytes is, never a different merge.
  • attributes.js — .gitattributes as git reads them: parse, wildmatch, precedence, macros.
  • rebase.js — rebaseBranch (per-commit or squash), cherryPick, revertCommit on the same merge primitive.
  • repo-node.ts — pushRepo(dir, url, { refspecs, all, tags, force, dryRun, thin }, opts): git push of a local repository with its history, no git binary (ply/repo-node, Node only). Reads .git itself (packs resolved in memory, loose objects hash-checked, packed and loose refs), one ls-refs, one receive-pack for every command; the pack is rev-list --objects <new> --not <remote refs>, delta-compressed by delta.ts (planDeltas: OFS_DELTA against the previous version of the same path, encodeDelta in the shape of git's diff-delta.c; thin REF_DELTA opt-in). A non-fast-forward is not_fast_forward before sending. Judged in test/push-repo.test.js. Issue #186.
  • refops.ts — createBranch, deleteBranch, createTag by URL: one receive-pack command each (an annotated tag carries its tag object); an existing ref is cas_conflict (expected zero, actual its tip). Judged in test/e2e-customer.test.js, which also runs README.md's quick start as written against a default and a stock git http-backend.

3. How the work was done (the process)

Git is the oracle. Every behaviour that git defines is decided by running git, not by reading docs or reasoning: git http-backend behind a tiny Node shim (test/server.js), a fault-injecting proxy (test/proxy.js), workerd for the Workers runtime, and git's own commands as judges — merge-tree --write-tree, merge, rebase, cherry-pick, revert, check-attr, rev-list --objects --filter, diff, fsck --strict, verify-commit, ssh-keygen -Y verify, index-pack, pack-objects. A second git (master, 2.56, built from source) is used wherever behaviour differs by version.

Evidence before claims. A claim goes into SAFETY.md only with the test that proves it. When a test fails, the first question is whether the oracle or the code is wrong; several "failures" were the oracle (e.g. git diff -U0 trims the common tail; a fast-forward merge where a merge commit was intended; a tie in parent order hiding a bug) and were fixed there, with the reason written down.

A bug becomes a test that fails on the old code. For the serious findings (merge base, criss-cross) the new tests were run against the previous code to confirm they catch it.

Property tests and fuzzers over random inputs, with seeds and repro commands. Every fuzzer prints a one-line repro (--seed, --case). npm test's fuzz stage (npm run test:fuzz-pinned, test/fuzz/pinned.mjs) is seeded by constants: the same cases every run, each entry printing its seed when it starts and, with a replay command, when it fails or hangs past the per-entry timeout. Discovery by a fresh seed (the wanted-tree finding came from a clock seed) runs in the nightly workflow only, through npm run test:fuzz-random, which prints its seed first; npm run coverage pins every seed so the coverage number is reproducible. A fuzzer feeds its git oracle from a file, not through execFileSync's input pipe, and gives the call a timeout: on node 24.18 / macOS under load that pipe call sometimes never returns for a 250 kB input (9 of about 36 000 calls with no ply code loaded, 0 of 13 000 from a file), which hung test/fuzz/codec.mjs.

Measure what is exercised. Fuzzers report what they reached (coverage buckets, "exercised" counters: union-resolved files, merge-attribute conflicts, Myers heuristic splits, mxcost cutoffs), so a green run is known not to be vacuous. A first run with zero differences is treated with suspicion until the counters show the hard paths ran.

Tree-shaking is asserted (test/bundle.test.js: esbuild metafile module lists, symbol greps, byte budgets).

Constraints in force: nothing was ever pushed to GitHub (the push side against GitHub is unexercised by design, pending real keys); the user's email address is used only to identify the user and is not written into the project.

4. Timeline

Earlier sessions (summarised; details in SAFETY.md):

  1. Core client: smart HTTP v2, partial fetch, streamed single-file edit, thin delta push, O(1) memory, OTLP telemetry, hostile-server safety (fault proxy, liar fuzz, hostile fuzz), exhaustive compliance against git.
  2. Learned from ivarvong/gitgrep → content-addressed pack cache for wants-only fetches (one bust retry if a cached pack fails verification) and the head hint (ls-refs overlapped with a speculative commit fetch; one internal retry on cas_conflict, linked by span).
  3. Telemetry overhaul ("all of it"): bounded events, CPU measured vs estimated, cache status, isolate cold marker, retry links; more FP throughout.
  4. Simplify without losing the general client: free functions, sideEffects: false, tree-shaking proven by the bundle test; coverage tooling and dev loop.
  5. Git fundamentals review; LFS-pointer guard; honest agent= string; SSH-signed commits (PROTOCOL.sshsig, Ed25519 via WebCrypto; verified by git and ssh-keygen; also under workerd).
  6. Branch operations needed for merging (scope narrowed by the user from "PRs" to git ops): create, fast-forward, merge, delete; then rebase, squash, cherry-pick, revert.
  7. SSH transport (removed 2026-10-31; history on branch archive/ssh).
  8. Layer review and refactor: the transport contract, operation(), the error table split, I26 (layering as a tested property).

This session (2026-10-19 → 20):

  1. Line-level merging: ported git's xdiff merge (histogram, compaction, xdl_do_merge, refinement). Found and fixed: rebase replaying against a blob that only existed in the pending push; duplicate objects in a pack ("tried to add buffer twice"); rebase keeping commits that became empty (git: --empty=drop); rename naming at targets and a G1 hole. Later: G1's deleted side is read off the three trees (a both-sides delete in a subtree the sides share, or under a file turned directory, was unseen); the shared subtrees' losses arm G1 only when a side added or removed a path of its own, and the gate reads the smaller set first (the shared subtrees, or the directories one side alone changed), 8 siblings at a time, each tree fetched once (test/merge-renames.test.js, counting transport; cost in SAFETY.md).
  2. Version skew: the remaining conservative answers all came from git 2.54's compaction re-diff. Built git master as a second oracle; ported classic Myers; made it the rediff option (myers ≡ git ≥ 2.54, skip ≡ older, undecided = safe for any). Double dissociation measured. Pinned the hash-chain limit (64 merges, 65 → git fails) exactly.
  3. Protocol model gap (fuzzer's model, not the client): NOT_USER_GIVEN semantics for wanted trees under tree:N; modelled from git's source, 1,080 targeted cases equal rev-list.
  4. Export (zip) after a fully green run and 100% coverage.
  5. "Where are our gaps?" → found and fixed two silent wrong-merge bugs in merge-base selection (criss-cross, and the summed-distance trap on ordinary histories); .gitattributes support (measured that the attribute source is version-dependent → explicit option); merge drivers binary/-merge/union with git's full refinement.
  6. This export.

5. Decisions and why

DecisionWhy (evidence)
readLog is first-parent, not git log's date orderthe first limit first-parent commits are exactly inside a deepen limit window and truncated is the last commit's parent count, with no date heuristics; it is the history a branch view shows (one merge per landed branch). A date-ordered git log -n would need the whole window ranked by committer date (issue #56)
readLog's window is held to the deepen contract like the merge-base walk'sa cut graph without shallow lines (or a foreign commit) is pack_corrupt, never a shorter or padded log: 90 cells in test/log.test.js, 50 of them predicted pack_corrupt from git's own graph
Transport = plain record with two methods; HTTP implements itoperations never branch on transport; an SSH transport was added (and removed 2026-10-31, archive/ssh) with zero changes above the contract
Errors: one table, codes validated at throwretry policy is data; tests check docs ↔ table ↔ literals in source stay aligned
After an acknowledged push, any read-back failure → ambiguousthe push may have landed; a transport code would invite a wrong retry (I25)
Pack cache keyed by sha256(url, credential, body), wants-onlymeasured against GitHub: wants-only fetches are deterministic; the store side is verified while streaming (O(chunk) state: framing, no ERR, pack trailer SHA-1), a HIT is served unparsed and the client's parse is the check: poison → bust + one retry
Merge = tree merge + ported line merge, never "close enough"clean results must equal git merge-tree byte for byte; anything uncertain is a conflict (superset)
Line merge work is budgeted: maxMergeWork steps per merged file (2^32 default, 2^26 in WORKER_LIMITS)git's histogram diff is O(n²) on inputs well under the byte cap; over the budget is a conflict (too large to merge), the one place we conflict where git merges cleanly (I27). Default: what git merges in about 5 s here, pair swaps and bulk edits both (2^31 refused a 130,000-line bulk edit git merges in 4.1 s); Worker: about 1 s of counted work
No rename detection; rename guards insteadporting merge-ort's rename detection is large; guards make every rename shape a conflict, never a wrong tree
rediff option, default myersgit changed behaviour in 2.54; both are git; default follows current git, skip/undecided documented
Merge base = git's best common ancestors, deepen until complete; criss-cross refusedthe old "closest" rule produced wrong merges (shown); a single base cannot reproduce git's virtual base
attributes option, default nullmeasured: git 2.43 bare repos read HEAD's attributes, 2.46+ read none — there is no single "git's answer"; the caller names the source
Merge bundle budget 66 KB (edit bundle 44 KB)line merge ≈ 10.4 KB and attributes ≈ 5.5 KB are deliberate; the edit path carries neither (asserted)
Git's GPL test vectors (t3070) run once, not shippedlicence; the result (380/380) is recorded in SAFETY.md
pushTree takes files as data; the directory walk is a separate Node module (ply/tree-node); likewise checkoutTree yields streams and ply/checkout-node writes themsrc/ stays Web-APIs-only for Workers; the bundle test asserts node: appears only in tree-node.js and checkout-node.js
pushTree's "server has it" set = oids named by the parent's trees it readexact for one-commit deploys (push 2 == git rev-list --objects new ^old, 7 objects / 1,176 B on a 208-entry tree); objects only in older history are resent — correct, not minimal, no history walk
checkoutTree fetches every tree in one blob:none pack, then blobs by wants list3 requests for a 22-entry fixture vs 72 for readCommit → listTree → readFile per file; blob:none is the one filter Artifacts serves, so the same 3 there. A path still fetches the whole commit's trees (one round trip beats a tree:0 walk; bounded by maxPackBytes)
blob:none fallback: the caller's limits bind what it asked for; an extra over a limit is dropped, a malformed one fails the fetcha tree:0 server never sends the extras, so failing on an honest big sibling directory made the answer depend on the host (#46: curl on Artifacts, 2,135-entry tests/data vs a 2,000 limit, 413 on every page); a malformed object means the response is untrusted (#18). Dropped extras are refetched as wanted objects if ever read. SAFETY I30
Blob batches are buffered and split on a cap, not streameda multi-object pack cannot be cut into objects without inflating each one (zlib end is found only by inflating); a batch is bounded by the same caps as any buffered pack, and a blob alone streams in O(1) memory (still under maxPackBytes on the wire: since issue #36 every fetch is). A split re-downloads up to maxPackBytes — the cost of not knowing blob sizes in advance (trees do not carry them)
Every push refuses a pack over maxPackBytes before sendinga capped host (Artifacts: 64 MiB of request body, so set the limit a little under it for the command prefix) would otherwise take the upload and answer 4xx; limit_exceeded carries the measured size
A push deflates its pack twice and streams the second pass (#172)holding the pack cost about 4 L for a push of largest object L: the pack, its concat, and the request body's concat (+268 MB live for 64 MiB). Pass 1 measures, so the cap still refuses before the first byte. Pass 2 streams, and its trailer is written only if it matches pass 1. Peak +78 KB live (+82 to +95 KB with 64 KiB zlib slices, below). A pack of at most 16 KiB is still sent as bytes. SAFETY I41
The deflate decoder looks codes up 9 bits at a time, from a bit buffer read up to 24 bits ahead (after #172)the pull decoder #198 made the only one went a code bit per step and back to the pull per symbol: inflate() of 64 MiB of text 2.10 s CPU (2.04 s for the buffered decoder before it), parsePack of one 64 MiB blob 2.34 s. The table is filled from the count/symbol arrays puff's decode walks, so it answers what decode answers on every 9-bit lookahead, incomplete and over-subscribed codes included (held exhaustively to puff). Since the code-set check (huffComplete, zlib's inftrees rule) a dynamic block's over-subscribed or incomplete set is refused at its header, so of those only the two shapes zlib permits reach the table from a member: no codes at all, and one 1-bit code in the literal/length or distance set (test/inflate-codesets.test.js, held to node:zlib); a longer code falls back to decode. Bytes read ahead are not consumed and go back to the input before a byte boundary, at the end and with a full chunk, so output, consumed and refusals are unchanged in every chunking. Now 1.12 s and 1.25 s; 2,000 4 KiB trees 0.53 → 0.30 s; random (stored) 64 MiB 0.53 → 0.21 s; under workerd 64 MiB text 1.62 → 1.06 s. Two 1 KiB tables are the only new memory. node test/bench.mjs --tree <old> --tree <new>
A push's two deflate passes stay; each hashes with streamingSha1measured per pass (node, 64 MiB): about 3.0 s, almost all zlib (pure-JS SHA-1 of the compressed bytes 0.2 s for text, 0.6 s for random); the length pass 1 must know before the first byte exists only once zlib has run, so the pass cannot be cheaper without holding the pack. On Workers the hash is now the native DigestStream: under workerd 64 MiB text 5.19 → 4.89 s, random 5.43 → 4.90 s (one pass, buildPack, was 2.36 s). Feeding zlib 64 KiB slices (DEFLATE_SLICE) instead of 16 KiB: taken (2026-11-06). 64 MiB push CPU, node (npm run bench, 5 runs, median): text 5,504 → 5,063 ms, random 5,190 → 4,791 ms; workerd (one process, trees alternated, 24 rounds, paired median): text −225 ms of 4,420, random −115 ms of 3,530 (3–5%, not the 6–8% an earlier process-per-run measurement showed). Cost: one slice's deflate output in flight, 81,940 B where it was 32,776 B (SAFETY I41). A cheaper pass-2 comparison than SHA-1 was measured and not taken: SAFETY.md, measured negative results
Tree-entry name checks and parseTree allocate nothing per entry they do not return (#207)prod profile of ply-web 59f39490: parseTree 8.3% of CPU, isDotFileName + isDotGitName 55% of it, GC 10%: a needle te.encoded per call, a [cp, len] tuple per character, closures, a mode decode + regex, take's Result, a concatenated key per directory, an Object.create + Object.assign per entry. Now needles are module constants, the checks read bytes in place, a name whose first byte begins no spelling and that holds no backslash exits after one switch and one indexOf, entries are one class with the buffer in a private field. Equivalence: the c91c9cf code is kept as the oracle (test/oracle/dotnames-ref.ts, parsetree-ref.ts); 38.8 M names (every byte string ≤ 3 bytes, ≤ 4 over 38 bytes, every spelling × head × tail with truncations/substitutions/ignorables, 10 M random) and 1 M random trees agree, 0 disagreements (test/fuzz/dotnames.mjs, parsetree.mjs). node test/bench-tree.mjs --runs 7 --passes 20, node 24.18, median [min–max] ns/entry: bun@2a99800 (13,785 entries) 2,238 [2,219–2,275] → 427 [424–429], llama.cpp@ec88c3c 2,310 → 534, ply@c91c9cf 2,849 → 899; bytes allocated per entry 2,251 → 316, 2,623 → 387, 2,677 → 379
A fetched tree is parsed once: the check's parse is the reader's (#208)prod profile: 48% of parseTree's CPU was assertFetched, the rest the reader parsing the same bytes again. assertFetched now answers a tree with its entries; a single-object fetch carries them on the FetchedObject (tree, with the entry cap they were parsed under) and treeEntries hands that array to the reader when the reader's maxTreeEntries is that cap (parseTree reads no other limit, so it is the parse the reader would make), else parses under the reader's limits. Every tree is still checked before it is returned; a harvested or wide answer, and bytes from a caller's cache, are parsed by the reader as before. Readers: listTree/readFile walk, changes, pushTree's parent trees, merge's cachedTree, editFile. node test/bench-tree.mjs --op fetched (check + read per fetched tree, 7 runs × 20 passes) ns/entry, c91c9cf → #207 → #208: bun 4,301 → 830 → 448, llama.cpp 4,233 → 931 → 529; B/entry 4,503 → 646 → 354
The pure-JS SHA-1 keeps int32 state and a 16-word Int32Array schedule (#214); hashObject stays sync and pure JSprod: SHA-1 3.6% of pooled CPU, 10.3% of heap. A Uint32Array read of a word ≥ 2^31 boxed a heap number every round, and the 80-word schedule cost 64 extra stores and loads per block. npm run bench -- --runs 5 --case sha1|hashObject (node 24.18) median: 1 MiB 7.11 → 3.77 ms; hashObject of 2,000 4 KiB trees 59.8 → 30.5 ms. Held to node:crypto (test/sha1.test.js). crypto.subtle.digest measured and not taken: async and whole-buffer, ~10 µs a call in node (slower than pure JS below ~2 KiB, 13× faster at 1 MiB); hashObject is sync on every fetched object, so it would be an API break through singleObject and every caller. Streams already use Workers' native DigestStream
readFile/listTree ask for the commit with combine:blob:none+tree:K (K = levels the path needs, at most 2) where the server takes it (#218)each path level was a round trip waiting on the last (ply-web prod traces, 6f2897ff). Measured 2026-11-06: GitHub refuses tree:N>0 and combine (git's own words, reproduced with uploadpackfilter.tree.maxDepth=0 + combine.allow=false), Artifacts refuses every filter but blob:none, gitlab.com and codeberg.org take both; no host advertises ref-in-want. Commit + root + one level, 7 rounds, median ms, 3 × tree:0 → one combine fetch: gitlab-org/gitlab 716 → 249, linux stable 605 → 279, forgejo 649 → 341, gitlab-runner 663 → 221. K = 3 rejected: 145–566 KB and 0.8–1.6 s per fetch on those repos (K = 2: 20–55 KB). Steady-state requests (test/bench-rt.mjs, base 814922e → this): gitlab-runner readFile depth 1/2/3 4/5/6 → 3/3/4, listTree / and depth 1 3/4 → 2/2; forgejo depth 1/3/4 4/6/7 → 3/4/5; GitHub and Artifacts unchanged (GitHub 3 + n, Artifacts 3 at any depth). Learned per quirks record in a WeakMap (the Quirks JSON shape is unchanged), asked only once tree:0 is settled (so Artifacts and stock git never see it and #136's one refusal stands), first ask concurrent with the tree:0 commit fetch; a failed deep fetch is never used. SAFETY I42
maxPackBytes is enforced in v2Step (band 1 and band 2), not by each consumer; every object's zlib stream is capped at zlibBound(size) in spStep and inflatethe streamed blob paths had no body cap at all: endless empty stored blocks (5 bytes in, 0 out) were read at ~90 MiB/s until the request timer (shown: 793.6 MiB / 8 s). One choke point means a new consumer cannot forget the cap; the per-object bound (zlib's own deflateBound, since RFC 1951 has no ceiling) means a 10-byte blob costs ≤ 26 zlib bytes plus one chunk whatever maxPackBytes is, and a buffered pack's parse is linear in declared sizes
Redirects never followed (redirect: 'manual', 3xx → http); the credential scrub rides on the transport into every span and errorworkerd 2026-10-23 forwards Authorization on a cross-origin redirect (measured); a server can echo the token in ERR/band 2/band 3/ng text. SAFETY I6b, I6c (issue #15)
URL userinfo is sent as git sends it (<token>@ → Basic <token>:, :pass → :pass, percent-decoded to bytes), not refusedmeasured on git 2.50 behind a Basic-auth gate: no prompt before the first credentialed request, the header byte-equal in every form; a library cannot prompt, so a refused one is auth. SAFETY I6d (issue #32)
An explicit auth that sends nothing is bad_request; { user } alone is refused, not user:git's own credential API (a helper giving a username alone) asks for the password and sends nothing; a library cannot ask. { user, token: '' } is the explicit user:. Explicit fields go out as UTF-8 bytes, as git sends a helper's. SAFETY I6e (issue #40)
planPackBatches' crossBatchRef stays opt-in, and its cost when on is accepted (2026-11-11)a delta whose base is in an earlier batch is written as REF_DELTA, a thin pack whether a host takes is the host's to say. With it off, planning is within 3% of main 8e3939d. With it on, each cut base's id is asked of resolved and checked: measured at 02374d4 against 8e3939d over 15 interleaved process rounds, 1.03 to 1.06 times on packs cut nowhere and 1.16 times (minimums 1.24) on an OFS pack in which nearly every push cuts a chain. README.md, planPackBatches
A rebase walks its branch past maxMergeDepth in windows, and its refusal's request count is stated and asserted (2026-11-11)the branch walk goes on from the cut frontier in windows of maxMergeDepth commits, one window held at a time as parent lists, so it holds no more than the merge-base search. A refusal costs 17 requests under WORKER_LIMITS (the merge-base search alone) and 26 under DEFAULT_LIMITS, both asserted in test/rebase.test.js on an onto history deeper than maxMergeDepth. README.md, rebaseBaseOf
limits.heldTimeoutMs bounds how long a consumer may hold a chunk (default 4 x idle, its own limit name); SAFETY I1 is plain "No hang" again (Ivar, 2026-11-11, #273 option B; #281)network idle is charged only while ply waits on the network, so before this nothing bounded a stalled consumer with default limits; the hold starts at the first clock read after the take (no clock read per chunk), so a cut is never early and at most min(idle, held) late
cachingFetch({ verifyBeforeStore: true }) stores only an answer verifyFetchAnswer accepted (wants present, objects hashed); without the flag the store path is main's eager put (2026-11-11, #270; #282)#269 had kept the option's name while it did less (a want-less answer with a valid trailer was stored); openBlob has no bust path, so a poisoned entry must never be written
maxResolveWork below maxInflatedTotal is refused at parse, null is refused like every key, and the resolver's ceiling is the parsed value (2026-11-11, #271; #280)the old Math.max raised a caller's tight budget silently, and telemetry disagreed with the configuration
A push whose body the transport never pulled is not a push: no settle reads, not counted, never lastPush: 'refused' (2026-11-11, #272; #283)the command list never left the process, so no ref can have moved; the fact is carried in the type ({ sent, result }), not in the error code
Regexes on remote text are linear-time, with no knownBad rows left in the regex inventory (2026-11-11, #274; #284)the fetch filter-refusal matcher was quadratic on remote input (344 ms at 56 KB)

6. Verification — how to rerun everything

Requirements: Node ≥ 22, git ≥ 2.43 on PATH, OpenSSH ssh-keygen for the signing suites, workerd (npm, pinned) for the Workers suite (skipped if absent).

Before every push, run the fast gate. It is what CI's lint + fuzz job and the registry test reject first: types and the ratchets, lint, the space registry (gate a, SPACES.md freshness), and the built dist importing the way consumers import it (scripts/dist-check.mjs builds into a temporary directory and imports ply from there; your dist/ is not rewritten, npm run build is what writes it). It needs no network. There are no git hooks: running it is the contributor's step, and notify-ply-web tells ply-web about a commit only after ci passed for that commit on main.

sh
npm run check             # typecheck + lint + registry test + dist check; run before every push

A ratchet baseline (scripts/throw-baseline.json, scripts/absence-baseline.json, test/space/ratchet.json, test/durations.json) moves the unsafe way only in a commit whose message has the line Ratchet-Change: <file>: <reason>. CI's node scripts/ratchet-guard.mjs compares each baseline with the base commit's and fails without that line; a squash merge must keep the line in its message, or main's own ci run fails. npm run spaces -- --ratchet only lowers test/space/ratchet.json. Adding a hole to it takes npm run spaces -- --ratchet --init --reason "<why>", which prints that line; with no reason it exits 1, lists the holes it would add and writes nothing.

The Git behaviour coverage matrix (test/behaviours/matrix.ts, rendered to COVERAGE.md by npm run coverage-matrix -- --md) counts server behaviours, not git commands: each has an area, the ply ops and tests that exercise it (grepped by test/behaviours/matrix.test.js, never asserted from memory), and a status of unsupported, implemented or verified. Verified carries evidence by kind (ply against git http-backend in this suite, ply, stock git or hand-built wire probes against Artifacts in a recorded run listed in test/behaviours/runs.ts); the types make a verified row without evidence impossible. A change that adds or retires a behaviour, a test, or a run updates the matrix and regenerates COVERAGE.md in the same commit.

Differential probes turn matrix behaviours into scenarios (plans, types and the pure evaluator in src/probe-*.ts, exported as ply/probes so ply-web's workbench runs the same plans from a Worker; the runner in test/probes/): a pure typed plan per <behaviour>#<variant> (a seed of deterministic objects, so every target holds the same ids; pushes or concurrent pushes; an optional fault: hold the receive-pack request while another writer moves refs, or cut the server's answer) and the outcome git http-backend is expected to give. Each scenario runs by ply and by stock git against git http-backend configured to Artifacts' known policy (test/probes/probes.test.js, in CI) and, opt-in and private, against Artifacts (npm run probes -- --artifacts --out <dir outside the repo>). Stock git judges every run (ls-remote, mirror clone, fsck --strict, rev-list --parents, merge-base) and src/probe-evaluate.ts compares runs on meaning: outcome and error class, refs and HEAD, reachable objects, trees, ancestry, and whether a push a client called landed is in the refs. Expected differences live in src/probe-allow.ts, each with its reason; anything else is a finding, classified by triangulation (a server bug only when ply is right on the reference and stock git reproduces it on the server), and an unused allow entry fails the run. The repository keeps run ids and check names (runs.ts), never verdicts.

sh
npm ci
npm test                  # 113 deterministic tests + 16 fuzz stages (≈29 min)
npm run coverage          # line coverage of src/, seeds pinned (expects 100%)
npm run fuzz:long         # the long campaigns
npm run test:push-full    # the full push matrix (1,400 cells)

Individual oracles (each prints a repro on failure):

WhatCommand
Line merge vs git merge-treenode test/fuzz/linemerge.mjs --n 2000 --seed 1 [--runs\|--big] [--driver union\|binary] [--rediff myers\|skip\|undecided]
Diff hunks vs git diff --histogramnode test/fuzz/diffscript.mjs --n 200 [--snake\|--huge]
.gitattributes vs git check-attrnode test/fuzz/attributes.mjs --n 200 --seed 1
Wanted-tree filter model vs rev-listnode test/fuzz/usergiven.mjs --n 60 --seed 500
Merge property (histories, renames, attributes)MERGE_SEED=11 MERGE_CASES=120 npx vitest run -t property test/merge.test.js
Rebase/cherry-pick propertyREBASE_SEED=1 REBASE_CASES=60 npx vitest run -t property test/rebase.test.js
Protocol product spacenode test/fuzz/protocol.mjs --n 200 --seed 32
CPU of inflate, parsePack and the push writer, old tree against new (interleaved, one process per case)npm run bench -- --runs 5 --tree <older checkout> --tree .
parseTree ns and bytes allocated per entry over every tree of a repository, old tree against newnode test/bench-tree.mjs --runs 7 --corpus <repo>@<rev> --tree <older checkout> --tree .
Tree-entry name checks and parseTree against the pre-#207 code (test/oracle/)node --import ./test/ts-resolve.mjs test/fuzz/dotnames.mjs (38.8 M names) and test/fuzz/parsetree.mjs --n 1000000

The oracle's version picks the matching mode automatically (test/gitversion.js). To judge against another git: GIT_BIN=/path/to/git node test/fuzz/linemerge.mjs … (and diffscript, attributes, linemerge.test.js).

Building git master as the second oracle (what was used here, commit 47fdf220, "2.56"):

sh
git clone --depth 1 https://github.com/git/git.git && cd git
make -j NO_CURL=1 NO_OPENSSL=1 NO_EXPAT=1 NO_GETTEXT=1 NO_TCLTK=1 NO_PERL=1 NO_PYTHON=1 git

Checks that only CI runs, and how to run them locally

On 2026-11-11 every red head of review train 2 was red for a check that npm run check and a branch's own test files do not run: gate c claims (three rounds), surviving mutants (three rounds), git 2.56 against a laptop's git, the fail-site recorder, Linux git init defaults, and zlib output sizes. Each can be reproduced before a push.

Gate c claims for one space. The registry (test/space/registry.js) gives each fail site in src/ to a space; gate c fails when a space's runners ran and none of them reached a site given to it. To judge one space, record over every runner file that space lists in the registry, then judge the records:

sh
rm -rf .failsites
PLY_FAILSITES=.failsites NODE_OPTIONS=--import=./test/space/failsite-hook.mjs npx vitest run test/space/describe-pack.test.js
PLY_FAILSITES=.failsites PLY_FAILSITES_GATE=1 npx vitest run test/space/failsites.test.js

npm test records and judges the whole suite the same way. A space's runners are its runners list (node -e "import('./test/space/registry.js').then(m => console.log(m.SPACES.find(s => s.name === 'describe-pack').runners))"); record all of them, because a claim is judged as soon as any one of them is in the records. On a partial record like this one, only the claims test counts: gate c: every fail site the registry gives a merged space is reached by that space must pass. The failsites test fails, since most of src/ was not run, and that failure means nothing here; the Linux test is skipped off Linux (macOS hides cells, and CI decides). Reached means a recorded stack passed through the site's frame (file:line:col, frameOf in test/space/holes.js) while one of the space's runners ran. It does not mean a test read the error: observed reports reached sites whose code no test read, and is not judged. To tell whether a claim hole is new, record and judge a checkout of main the same way, with the same runner files, and compare the two claims lists.

The mutation gate. Only the test files in MUTATION_FILES (vitest.stryker.config.ts) run under Stryker, so only they kill mutants. A test elsewhere that pins a line does not count: a mutant it would kill survives in CI. A test written to kill a mutant goes in one of those files, or its file is added to the list. A narrow run on a line range uses the repository's config (stryker.conf.json, which names vitest.stryker.config.ts):

sh
npx stryker run --mutate 'src/format.ts:1-30' --concurrency 4

The dry run first runs every file in MUTATION_FILES (about 30 s here); the mutants of the range then run against the tests that cover them, and the clear-text report lists each survivor. Which survivors are fatal is decided in scripts/mutate.mjs only: a surviving or uncovered mutant in a parser function (the PARSER pattern there) or in a bounds file (BOUNDS_FILES: src/bytes, src/format, src/inflatestream, src/packstream) fails the build; others are reported. An equivalent mutant is allowed only as // Stryker disable next-line <Mutator>: <reason> on its line. node scripts/mutate.mjs --check judges an existing reports/mutation/mutation.json with CI's rules; on a narrow run it judges only the mutants that ran. CI runs the gate out of band (mutate.yml, 20 shards; node scripts/mutate.mjs --budget in job lint + fuzz fails when the heaviest planned shard passes 60% of the job limit), so a pull request's ci can be green while mutate is still red; merge after both.

CI's environment, not a laptop's. CI runs Node 22 on ubuntu-latest, whose git is 2.56; a laptop often has another git (2.50 here) and Node 24. What differed:

  • Tests that read the git version (test/gitversion.js) take another branch on 2.56. Run them under a 2.56 build (the build above): put its directory first on PATH, or GIT_BIN= for the oracles that take it.
  • git 2.56 started background maintenance after a push, and a repack renamed packs under a test's fsck. Every git the suite starts is test/git-shim/git (first on PATH from vitest.config.ts), which turns auto-maintenance off; a test that starts git some other way must go through it, or through makeBareRepo / initRepo, which set the keys. test/harness-maintenance.test.js observes this.
  • Every CI test shard runs under the fail-site recorder (PLY_FAILSITES and the NODE_OPTIONS hook). It changes stacks and the code accessor of ply's errors; a test that counts bytes, frames or properties of an error can pass bare and fail there. Run a changed test once under the recorder (the first command above).
  • Linux git init writes no core.ignorecase or core.precomposeunicode; macOS writes both. A test that compares a repository's config, or depends on name folding, must set them or not read them.
  • git's zlib and node's zlib can differ by a byte for the same objects, and which is smaller depends on the commit timestamp. A test must not assume one pack is smaller than another; it measures both and sets its bound from the larger (the maxPackBytes boundary in test/push-repo-fs.test.js).
  • test/workflows.test.js pins the workflow files' text. After editing a workflow, run it and the linters CI runs:
sh
npx vitest run test/workflows.test.js
actionlint
uvx zizmor@1.31.1 --offline --persona regular .github/workflows   # CI's version; config in .github/zizmor.yml

A pass is the test file green, actionlint silent with exit 0, and zizmor's "No findings to report".

Reviewing a change. Run every test file that imports a changed src module, not only the test files the change touched. A closure check that ran only a branch's own tests passed while about 25 tests elsewhere failed (a change to how spanOn treats a tracer). To list them: git diff --name-only main... -- src for the modules, then git grep -l for each module's import path under test/. When a train head is red, bisect along its merges.

Benchmarking a perf bar. On a loaded machine, a number from one process run is noise. Interleave processes of main, the candidate (head) and a second copy of main (A/A), one sample per process, many rounds. The verdict is (head/main) / (aa/main): a candidate is faster only by more than the A/A arm moved. Sample memory peaks every 2 ms; a coarser interval misses them. Use realistic input sizes: a win measured on 1 to 8 byte inputs was an artifact that did not hold at real sizes.

7. Numbers (2026-10-20)

  • Bundles (esbuild, minified): edit 42.6 KB (16.5 KB gz); merge 63.9 KB; full 116 KB.
  • Line merge: 0 wrong across >40,000 random merges against 2.43 and 2.56; union 5,280 byte-identical; diff hunks identical on >1,400 diffs incl. 947 Myers heuristic splits and 160+ mxcost cutoffs.
  • Attributes: >40,000 paths equal to check-attr on both versions; 380/380 git wildmatch vectors.
  • Merge property runs: 0 wrong across seeds 11–15 with merge-commit and criss-cross histories and random attributes.
  • Coverage: 100% of 2,461 lines.

8. Open gaps and next steps

In order of value:

  1. Real keys: a GitHub push run (receive-pack policy layer, protected branches, the exact ng wording the cas_conflict classifier relies on) and an observed run on the Workers edge (CPU, subrequest and memory limits).
  2. Rename detection (merge-ort's): today every rename shape is a conflict — the largest source of conservative conflicts on real refactors.
  3. Virtual merge base for criss-cross histories (refused today).
  4. Conflict output: we refuse rather than produce markers/partial results; no resolution flow.
  5. Server-side configuration ply cannot see: info/attributes, custom merge drivers, merge.default.
  6. Rebase niceties: patch-id skipping of already-upstream commits; merges in the chain (not_linear today).
  7. Protocol/transport limits: SHA-256 repositories; servers without protocol v2; no SSH transport (removed 2026-10-31; history on branch archive/ssh).
  8. Features not built: annotated tags, multi-file edits on top of the parent tree (pushTree replaces the whole tree), LFS beyond the guard, merges of files over 8 MiB, a ply push <dir> CLI and .gitignore reading for pushDir; for checkoutTree: submodule recursion, .gitattributes (eol/filters) and LFS smudging, sparse fetching of only path's trees, and checkout into a non-empty directory.

9. What is in this export

ply-export/
  ply/                 the project (no node_modules, no fuzz-out traces) — run `npm ci`
    README.md              user guide
    SAFETY.md              invariants, findings, run record, what is not proven
    CONTEXT.md             this file
    src/  test/  package.json  package-lock.json
  process/
    README.md              index of the logs and experiments below
    logs/                  the final npm test run, the coverage report, the 20k line-merge campaign
    experiments/           the one-off scripts behind the findings (criss-cross, trap, chain limit, …)
1# ply — project context
3This is the hand-over document: what ply is, how it is put together, how it was built, every decision that matters and
4the evidence behind it, how to verify all of it again, and what is still open. README.md is the user-facing guide;
5SAFETY.md is the invariant ledger (each claim, the test that proves it, the findings and the run record). This file is the
6map between them.
8As of 2026-10-20: 35 modules in `src/` (3,606 lines), 113 deterministic tests in 19 suites plus 16 fuzz stages in
9`npm test` (≈29 min, all green), 100% line coverage of `src/` (2,461 lines, `npm run coverage`, reproducible seeds).
11---
13## 0. The name
15**ply** — to *ply* is to work a route steadily, back and forth (a ferry plies the river): this client works the wire
16between your code and a repository it never brings home. A *ply* is also one thin layer of something larger — a
17one-line delta on a 10 GB repository. Renamed from *gitwire* on 2026-10-23: git's trademark policy (git-scm.com/about/
18trademark, §2.3) does not allow "Git" "as a syllable in a new word or as part of a portmanteau" for a third-party
19product; "… for Git" is the permitted form, so the tagline is *"ply — a git client for anywhere"*. The bare npm name
20`ply` is an abandoned 0.1.0 package: publish under a scope. Everything was renamed — package name, user agent
21(`ply/0.1.0`, `agent=ply/…`), request headers (`x-ply-cache`, `x-ply-pack-cache`), span attributes (`ply.*`),
22`isPlyCode`, environment variables (`PLY_*`) — so traces and server logs say ply.
24## 1. What ply is
26A zero-dependency JavaScript git client for Cloudflare Workers (and Node), speaking git's own wire protocols — smart
27HTTP (protocol v2 for fetch, v0 receive-pack for push) — with no clone, no working tree and no local object store.
29What it does:
31- **Edit one file in a huge repository in O(1) memory**: ls-refs → fetch the commit (`tree:0`, depth 1) → one tree per
32 path level → stream the blob through inflate, find/replace and SHA-1 without holding it → push a thin pack whose blob is
33 a delta against the old one (hundreds of bytes for a one-line change in a multi-GB repo). Compare-and-swap on the ref:
34 a lost race is `cas_conflict`, never a clobber. Optional SSH commit signing, verification read-back, LFS-pointer guard.
35- **Land branches with git operations only**: create (CAS), fast-forward, merge commit, delete; rebase, squash,
36 cherry-pick, revert — all on one three-way tree merge plus a line merge ported from git's xdiff, so clean results are
37 byte-for-byte what `git merge-tree` produces.
38- **A general client** (ls-refs, fetch/negotiate, push, discovery) for everything else, tree-shakeable so an edit-only
39 Worker carries ~42.6 KB minified.
40- **Telemetry**: OTLP/JSON spans for every wire round trip and every operation, with bounded events, measured vs
41 estimated CPU, cache status, retry links.
43Design constraints, held throughout: zero runtime dependencies; Web APIs only in `src/` (no `node:` imports — checked by
44the bundle test); pragmatic functional style (pure step functions and state machines, effects at the edges, free
45functions over classes so bundlers can drop what you do not import); every wire-sized allocation bounded by `limits.js`;
46every error typed (`errors.js`, one table with a retry policy).
48## 2. Map of the code
50Layers, bottom up (each depends only on the ones below):
52| Layer | Modules | Notes |
53|---|---|---|
54| Codecs (pure) | `bytes`, `pktline`, `inflate`, `sha1`, `objects`, `deltaenc`, `pack`, `packresolve`, `validate` | streaming where it matters (inflate reports bytes consumed; SHA-1 is incremental) |
55| Errors & limits | `errors`, `errors-doc`, `limits` | 27 codes, each with a retry policy (`yes`/`fresh`/`no`/`read`); docs kept apart so the edit bundle does not carry them |
56| Transport contract | `transport`, `http` | a transport is a plain record `{ frames(service, body, span, hints), advertise(service, span) }`; one implementation (HTTP), no dispatch |
57| Protocol | `lsrefs`, `fetch`, `fetchpack`, `negotiate`, `push`, `receivecaps`, `discover`, `packcache` | v2 fetch as a pure state machine (`v2Init`/`v2Step`); push report parsing pure |
58| Operations | `operation`, `edit`, `tree`, `checkout`, `log`, `merge`, `linemerge`, `attributes`, `rebase`, `sign` | `operation()` owns the root span, transport, outcome and the "after an acknowledged push, any read-back failure is `ambiguous`" rule |
59| Entry points | `edit.js` (edit-only bundle), `tree.js` (whole-tree push), `checkout.js` (whole-tree read), `index.js` (everything), `client.js` (`GitHttp` convenience wrapper); Node-only adapters outside every Workers bundle: `tree-node.js` and `checkout-node.js` (`node:fs`) | `sideEffects: false` |
60| Telemetry | `otel` | minimal OTLP/JSON tracer; exporters pluggable (`test/jsonl.mjs` for files) |
62Key operation modules:
64- `edit.js` — `editFile(url, spec, opts)`: find/replace in one path, create-on-branch (`ref` + `base`), head hint (ls-refs in
65 parallel with a speculative commit fetch), pack cache, signing, verify, LFS guard.
66- `tree.js` — `pushTree(url, spec, opts)`: a caller-given file set (object, Map or iterable of `{ path, mode, data|oid }`,
67 symlinks 120000, gitlinks 160000) as one commit, with editFile's head hint, fresh `cas_conflict` retry, `strictHead`,
68 `verify`/`ambiguous`, signing and tree cache. Thin repeat pushes: the parent's trees are read only where the new tree
69 differs, known blobs are not sent, changed trees go as `REF_DELTA`s. `tree-node.js` (`filesFromDir`, `pushDir`) is the
70 Node directory walk. Issue #11.
71- `checkout.js` — `checkoutTree(url, { ref, path }, opts)`: the mirror of pushTree. ls-refs → one `blob:none` fetch of the
72 commit (every tree) → blobs in `batch`-sized wants lists (buffered, delta-resolved, keyed by computed hash; an unwanted
73 object is `pack_corrupt`), a batch over a cap split in half down to single blobs that stream through read.js's
74 `openBlob`; a host refusing many wants gets one per request (`quirks.blobWants`). Returns `entries` and a `files` async
75 iterator. `checkout-node.js` (`writeFiles`, `checkoutDir`) writes a directory as `git clone` lays it out and never
76 writes through a symlink. Issue #12.
77- `log.js` — `readLog(url, { ref, limit }, opts)`: first-parent history (`git rev-list --first-parent -n`) from one
78 `deepen <limit>` fetch through `fetchPackTree0` (`tree:0`, else `blob:none` with the trees harvested). `logWindow` (pure)
79 holds the window to the #37 deepen contract (the want present; every commit an ancestor; a missing parent only under a
80 `shallow` line, else `pack_corrupt`) and sets `truncated` from the last commit's parents. No `path` filter. Issue #56.
81- `merge.js` — `mergeBranch`, `fastForward`, `deleteRef`, `mergeTrees` (trivial-merge rules per directory, only differing
82 subtrees fetched), `bestBases` (git's `merge-base --all` with a completeness check), rename guards, `blobMerger`,
83 `attributeDrivers`/`attributeTree`.
84- `linemerge.js` — port of git's xdiff as merge-ort drives it: histogram diff (+ hash-chain limit), classic Myers fallback
85 (prepare-stage trim/cleanup, `xdl_split` heuristics), group compaction (+ git 2.54's shifted-group re-diff), build
86 script, `xdl_do_merge`, conflict refinement and simplification, fill (clean and `merge=union`). Its diff work is
87 counted (every inner-loop iteration of the histogram scan and the classic fallback, on the module's own counter)
88 against `limits.maxMergeWork`, absolute per merged file: past it the file is a conflict with reason `too large to
89 merge`, as one over `maxMergeBlobBytes` is, never a different merge.
90- `attributes.js` — `.gitattributes` as git reads them: parse, wildmatch, precedence, macros.
91- `rebase.js` — `rebaseBranch` (per-commit or squash), `cherryPick`, `revertCommit` on the same merge primitive.
92- `repo-node.ts` — `pushRepo(dir, url, { refspecs, all, tags, force, dryRun, thin }, opts)`: `git push` of a local
93 repository with its history, no git binary (`ply/repo-node`, Node only). Reads `.git` itself (packs resolved in
94 memory, loose objects hash-checked, packed and loose refs), one ls-refs, one receive-pack for every command; the pack is
95 `rev-list --objects <new> --not <remote refs>`, delta-compressed by `delta.ts` (`planDeltas`: OFS_DELTA against the
96 previous version of the same path, `encodeDelta` in the shape of git's diff-delta.c; thin REF_DELTA opt-in). A
97 non-fast-forward is `not_fast_forward` before sending. Judged in `test/push-repo.test.js`. Issue #186.
98- `refops.ts` — `createBranch`, `deleteBranch`, `createTag` by URL: one receive-pack command each (an annotated tag carries
99 its tag object); an existing ref is `cas_conflict` (expected zero, actual its tip). Judged in `test/e2e-customer.test.js`,
100 which also runs README.md's quick start as written against a default and a stock `git http-backend`.
102## 3. How the work was done (the process)
104**Git is the oracle.** Every behaviour that git defines is decided by running git, not by reading docs or reasoning:
105`git http-backend` behind a tiny Node shim (`test/server.js`), a fault-injecting proxy (`test/proxy.js`), `workerd` for the Workers runtime, and git's own commands as judges — `merge-tree --write-tree`,
106`merge`, `rebase`, `cherry-pick`, `revert`, `check-attr`, `rev-list --objects --filter`, `diff`, `fsck --strict`,
107`verify-commit`, `ssh-keygen -Y verify`, `index-pack`, `pack-objects`. A second git (master, 2.56, built from source) is
108used wherever behaviour differs by version.
110**Evidence before claims.** A claim goes into SAFETY.md only with the test that proves it. When a test fails, the first
111question is whether the oracle or the code is wrong; several "failures" were the oracle (e.g. `git diff -U0` trims the
112common tail; a fast-forward merge where a merge commit was intended; a tie in parent order hiding a bug) and were fixed
113there, with the reason written down.
115**A bug becomes a test that fails on the old code.** For the serious findings (merge base, criss-cross) the new tests
116were run against the previous code to confirm they catch it.
118**Property tests and fuzzers over random inputs, with seeds and repro commands.** Every fuzzer prints a one-line repro
119(`--seed`, `--case`). `npm test`'s fuzz stage (`npm run test:fuzz-pinned`, test/fuzz/pinned.mjs) is seeded by constants:
120the same cases every run, each entry printing its seed when it starts and, with a replay command, when it fails or hangs
121past the per-entry timeout. Discovery by a fresh seed (the wanted-tree finding came from a clock seed) runs in the nightly
122workflow only, through `npm run test:fuzz-random`, which prints its seed first; `npm run coverage` pins every seed so
123the coverage number is reproducible. A fuzzer feeds its git oracle from a file, not through `execFileSync`'s `input`
124pipe, and gives the call a timeout: on node 24.18 / macOS under load that pipe call sometimes never returns for a 250 kB
125input (9 of about 36 000 calls with no ply code loaded, 0 of 13 000 from a file), which hung test/fuzz/codec.mjs.
127**Measure what is exercised.** Fuzzers report what they reached (coverage buckets, "exercised" counters: union-resolved
128files, merge-attribute conflicts, Myers heuristic splits, mxcost cutoffs), so a green run is known not to be vacuous.
129A first run with zero differences is treated with suspicion until the counters show the hard paths ran.
131**Tree-shaking is asserted** (`test/bundle.test.js`: esbuild metafile module lists, symbol greps, byte budgets).
133**Constraints in force**: nothing was ever pushed to GitHub (the push side against GitHub is unexercised by design, pending
134real keys); the user's email address is used only to identify the user and is not written into the project.
136## 4. Timeline
138Earlier sessions (summarised; details in SAFETY.md):
1401. Core client: smart HTTP v2, partial fetch, streamed single-file edit, thin delta push, O(1) memory, OTLP telemetry,
141 hostile-server safety (fault proxy, liar fuzz, hostile fuzz), exhaustive compliance against git.
1422. Learned from ivarvong/gitgrep → content-addressed pack cache for wants-only fetches (one bust retry if a cached pack
143 fails verification) and the head hint (ls-refs overlapped with a speculative commit fetch; one internal retry on
144 `cas_conflict`, linked by span).
1453. Telemetry overhaul ("all of it"): bounded events, CPU measured vs estimated, cache status, isolate cold marker, retry
146 links; more FP throughout.
1474. Simplify without losing the general client: free functions, `sideEffects: false`, tree-shaking proven by the bundle
148 test; coverage tooling and dev loop.
1495. Git fundamentals review; LFS-pointer guard; honest `agent=` string; SSH-signed commits (PROTOCOL.sshsig, Ed25519 via
150 WebCrypto; verified by git and ssh-keygen; also under workerd).
1516. Branch operations needed for merging (scope narrowed by the user from "PRs" to git ops): create, fast-forward, merge,
152 delete; then rebase, squash, cherry-pick, revert.
1537. SSH transport (removed 2026-10-31; history on branch archive/ssh).
1548. Layer review and refactor: the transport contract, `operation()`, the error table split, I26 (layering as a tested
155 property).
157This session (2026-10-19 → 20):
1599. Line-level merging: ported git's xdiff merge (histogram, compaction, `xdl_do_merge`, refinement). Found and fixed:
160 rebase replaying against a blob that only existed in the pending push; duplicate objects in a pack ("tried to add
161 buffer twice"); rebase keeping commits that became empty (git: `--empty=drop`); rename naming at targets and a G1 hole. Later: G1's deleted side is read off the three trees (a both-sides delete in a
162 subtree the sides share, or under a file turned directory, was unseen); the shared subtrees' losses arm G1 only when a
163 side added or removed a path of its own, and the gate reads the smaller set first (the shared subtrees, or the
164 directories one side alone changed), 8 siblings at a time, each tree fetched once (`test/merge-renames.test.js`,
165 counting transport; cost in SAFETY.md).
16610. Version skew: the remaining conservative answers all came from git 2.54's compaction re-diff. Built git master as a
167 second oracle; ported classic Myers; made it the `rediff` option (`myers` ≡ git ≥ 2.54, `skip` ≡ older, `undecided`
168 = safe for any). Double dissociation measured. Pinned the hash-chain limit (64 merges, 65 → git fails) exactly.
16911. Protocol model gap (fuzzer's model, not the client): `NOT_USER_GIVEN` semantics for wanted trees under `tree:N`;
170 modelled from git's source, 1,080 targeted cases equal `rev-list`.
17112. Export (zip) after a fully green run and 100% coverage.
17213. "Where are our gaps?" → found and fixed two silent wrong-merge bugs in merge-base selection (criss-cross, and the
173 summed-distance trap on ordinary histories); `.gitattributes` support (measured that the attribute source is
174 version-dependent → explicit option); merge drivers `binary`/`-merge`/`union` with git's full refinement.
17514. This export.
177## 5. Decisions and why
179| Decision | Why (evidence) |
180|---|---|
181| readLog is first-parent, not git log's date order | the first `limit` first-parent commits are exactly inside a `deepen limit` window and `truncated` is the last commit's parent count, with no date heuristics; it is the history a branch view shows (one merge per landed branch). A date-ordered `git log -n` would need the whole window ranked by committer date (issue #56) |
182| readLog's window is held to the deepen contract like the merge-base walk's | a cut graph without `shallow` lines (or a foreign commit) is `pack_corrupt`, never a shorter or padded log: 90 cells in `test/log.test.js`, 50 of them predicted `pack_corrupt` from git's own graph |
183| Transport = plain record with two methods; HTTP implements it | operations never branch on transport; an SSH transport was added (and removed 2026-10-31, archive/ssh) with zero changes above the contract |
184| Errors: one table, codes validated at throw | retry policy is data; tests check docs ↔ table ↔ literals in source stay aligned |
185| After an acknowledged push, any read-back failure → `ambiguous` | the push may have landed; a transport code would invite a wrong retry (I25) |
186| Pack cache keyed by sha256(url, credential, body), wants-only | measured against GitHub: wants-only fetches are deterministic; the store side is verified while streaming (O(chunk) state: framing, no ERR, pack trailer SHA-1), a HIT is served unparsed and the client's parse is the check: poison → bust + one retry |
187| Merge = tree merge + ported line merge, never "close enough" | clean results must equal `git merge-tree` byte for byte; anything uncertain is a conflict (superset) |
188| Line merge work is budgeted: `maxMergeWork` steps per merged file (2^32 default, 2^26 in `WORKER_LIMITS`) | git's histogram diff is O(n²) on inputs well under the byte cap; over the budget is a conflict (`too large to merge`), the one place we conflict where git merges cleanly (I27). Default: what git merges in about 5 s here, pair swaps and bulk edits both (2^31 refused a 130,000-line bulk edit git merges in 4.1 s); Worker: about 1 s of counted work |
189| No rename detection; rename guards instead | porting merge-ort's rename detection is large; guards make every rename shape a conflict, never a wrong tree |
190| `rediff` option, default `myers` | git changed behaviour in 2.54; both are git; default follows current git, `skip`/`undecided` documented |
191| Merge base = git's best common ancestors, deepen until complete; criss-cross refused | the old "closest" rule produced wrong merges (shown); a single base cannot reproduce git's virtual base |
192| `attributes` option, default `null` | measured: git 2.43 bare repos read HEAD's attributes, 2.46+ read none — there is no single "git's answer"; the caller names the source |
193| Merge bundle budget 66 KB (edit bundle 44 KB) | line merge ≈ 10.4 KB and attributes ≈ 5.5 KB are deliberate; the edit path carries neither (asserted) |
194| Git's GPL test vectors (t3070) run once, not shipped | licence; the result (380/380) is recorded in SAFETY.md |
195| pushTree takes files as data; the directory walk is a separate Node module (`ply/tree-node`); likewise checkoutTree yields streams and `ply/checkout-node` writes them | `src/` stays Web-APIs-only for Workers; the bundle test asserts `node:` appears only in `tree-node.js` and `checkout-node.js` |
196| pushTree's "server has it" set = oids named by the parent's trees it read | exact for one-commit deploys (push 2 == `git rev-list --objects new ^old`, 7 objects / 1,176 B on a 208-entry tree); objects only in older history are resent — correct, not minimal, no history walk |
197| checkoutTree fetches every tree in one `blob:none` pack, then blobs by wants list | 3 requests for a 22-entry fixture vs 72 for readCommit → listTree → readFile per file; `blob:none` is the one filter Artifacts serves, so the same 3 there. A `path` still fetches the whole commit's trees (one round trip beats a tree:0 walk; bounded by `maxPackBytes`) |
198| blob:none fallback: the caller's limits bind what it asked for; an extra over a limit is dropped, a malformed one fails the fetch | a tree:0 server never sends the extras, so failing on an honest big sibling directory made the answer depend on the host (#46: curl on Artifacts, 2,135-entry tests/data vs a 2,000 limit, 413 on every page); a malformed object means the response is untrusted (#18). Dropped extras are refetched as wanted objects if ever read. SAFETY I30 |
199| Blob batches are buffered and split on a cap, not streamed | a multi-object pack cannot be cut into objects without inflating each one (zlib end is found only by inflating); a batch is bounded by the same caps as any buffered pack, and a blob alone streams in O(1) memory (still under `maxPackBytes` on the wire: since issue #36 every fetch is). A split re-downloads up to `maxPackBytes` — the cost of not knowing blob sizes in advance (trees do not carry them) |
200| Every push refuses a pack over `maxPackBytes` before sending | a capped host (Artifacts: 64 MiB of request body, so set the limit a little under it for the command prefix) would otherwise take the upload and answer 4xx; `limit_exceeded` carries the measured size |
201| A push deflates its pack twice and streams the second pass (#172) | holding the pack cost about 4 L for a push of largest object L: the pack, its concat, and the request body's concat (+268 MB live for 64 MiB). Pass 1 measures, so the cap still refuses before the first byte. Pass 2 streams, and its trailer is written only if it matches pass 1. Peak +78 KB live (+82 to +95 KB with 64 KiB zlib slices, below). A pack of at most 16 KiB is still sent as bytes. SAFETY I41 |
202| The deflate decoder looks codes up 9 bits at a time, from a bit buffer read up to 24 bits ahead (after #172) | the pull decoder #198 made the only one went a code bit per step and back to the pull per symbol: `inflate()` of 64 MiB of text 2.10 s CPU (2.04 s for the buffered decoder before it), `parsePack` of one 64 MiB blob 2.34 s. The table is filled from the count/symbol arrays puff's decode walks, so it answers what decode answers on every 9-bit lookahead, incomplete and over-subscribed codes included (held exhaustively to puff). Since the code-set check (`huffComplete`, zlib's inftrees rule) a dynamic block's over-subscribed or incomplete set is refused at its header, so of those only the two shapes zlib permits reach the table from a member: no codes at all, and one 1-bit code in the literal/length or distance set (`test/inflate-codesets.test.js`, held to node:zlib); a longer code falls back to decode. Bytes read ahead are not consumed and go back to the input before a byte boundary, at the end and with a full chunk, so output, `consumed` and refusals are unchanged in every chunking. Now 1.12 s and 1.25 s; 2,000 4 KiB trees 0.53 → 0.30 s; random (stored) 64 MiB 0.53 → 0.21 s; under workerd 64 MiB text 1.62 → 1.06 s. Two 1 KiB tables are the only new memory. `node test/bench.mjs --tree <old> --tree <new>` |
203| A push's two deflate passes stay; each hashes with `streamingSha1` | measured per pass (node, 64 MiB): about 3.0 s, almost all zlib (pure-JS SHA-1 of the compressed bytes 0.2 s for text, 0.6 s for random); the length pass 1 must know before the first byte exists only once zlib has run, so the pass cannot be cheaper without holding the pack. On Workers the hash is now the native `DigestStream`: under workerd 64 MiB text 5.19 → 4.89 s, random 5.43 → 4.90 s (one pass, buildPack, was 2.36 s). Feeding zlib 64 KiB slices (`DEFLATE_SLICE`) instead of 16 KiB: taken (2026-11-06). 64 MiB push CPU, node (`npm run bench`, 5 runs, median): text 5,504 → 5,063 ms, random 5,190 → 4,791 ms; workerd (one process, trees alternated, 24 rounds, paired median): text −225 ms of 4,420, random −115 ms of 3,530 (3–5%, not the 6–8% an earlier process-per-run measurement showed). Cost: one slice's deflate output in flight, 81,940 B where it was 32,776 B (SAFETY I41). A cheaper pass-2 comparison than SHA-1 was measured and not taken: SAFETY.md, measured negative results |
204| Tree-entry name checks and parseTree allocate nothing per entry they do not return (#207) | prod profile of ply-web 59f39490: parseTree 8.3% of CPU, isDotFileName + isDotGitName 55% of it, GC 10%: a needle `te.encode`d per call, a `[cp, len]` tuple per character, closures, a mode decode + regex, `take`'s Result, a concatenated key per directory, an Object.create + Object.assign per entry. Now needles are module constants, the checks read bytes in place, a name whose first byte begins no spelling and that holds no backslash exits after one switch and one indexOf, entries are one class with the buffer in a private field. Equivalence: the c91c9cf code is kept as the oracle (test/oracle/dotnames-ref.ts, parsetree-ref.ts); 38.8 M names (every byte string ≤ 3 bytes, ≤ 4 over 38 bytes, every spelling × head × tail with truncations/substitutions/ignorables, 10 M random) and 1 M random trees agree, 0 disagreements (test/fuzz/dotnames.mjs, parsetree.mjs). `node test/bench-tree.mjs --runs 7 --passes 20`, node 24.18, median [min–max] ns/entry: bun@2a99800 (13,785 entries) 2,238 [2,219–2,275] → 427 [424–429], llama.cpp@ec88c3c 2,310 → 534, ply@c91c9cf 2,849 → 899; bytes allocated per entry 2,251 → 316, 2,623 → 387, 2,677 → 379 |
205| A fetched tree is parsed once: the check's parse is the reader's (#208) | prod profile: 48% of parseTree's CPU was assertFetched, the rest the reader parsing the same bytes again. assertFetched now answers a tree with its entries; a single-object fetch carries them on the FetchedObject (`tree`, with the entry cap they were parsed under) and `treeEntries` hands that array to the reader when the reader's `maxTreeEntries` is that cap (parseTree reads no other limit, so it is the parse the reader would make), else parses under the reader's limits. Every tree is still checked before it is returned; a harvested or wide answer, and bytes from a caller's cache, are parsed by the reader as before. Readers: listTree/readFile walk, changes, pushTree's parent trees, merge's cachedTree, editFile. `node test/bench-tree.mjs --op fetched` (check + read per fetched tree, 7 runs × 20 passes) ns/entry, c91c9cf → #207 → #208: bun 4,301 → 830 → 448, llama.cpp 4,233 → 931 → 529; B/entry 4,503 → 646 → 354 |
206| The pure-JS SHA-1 keeps int32 state and a 16-word Int32Array schedule (#214); `hashObject` stays sync and pure JS | prod: SHA-1 3.6% of pooled CPU, 10.3% of heap. A Uint32Array read of a word ≥ 2^31 boxed a heap number every round, and the 80-word schedule cost 64 extra stores and loads per block. `npm run bench -- --runs 5 --case sha1|hashObject` (node 24.18) median: 1 MiB 7.11 → 3.77 ms; hashObject of 2,000 4 KiB trees 59.8 → 30.5 ms. Held to node:crypto (test/sha1.test.js). `crypto.subtle.digest` measured and not taken: async and whole-buffer, ~10 µs a call in node (slower than pure JS below ~2 KiB, 13× faster at 1 MiB); hashObject is sync on every fetched object, so it would be an API break through singleObject and every caller. Streams already use Workers' native `DigestStream` |
207| readFile/listTree ask for the commit with `combine:blob:none+tree:K` (K = levels the path needs, at most 2) where the server takes it (#218) | each path level was a round trip waiting on the last (ply-web prod traces, 6f2897ff). Measured 2026-11-06: GitHub refuses `tree:N>0` and `combine` (git's own words, reproduced with `uploadpackfilter.tree.maxDepth=0` + `combine.allow=false`), Artifacts refuses every filter but blob:none, gitlab.com and codeberg.org take both; no host advertises ref-in-want. Commit + root + one level, 7 rounds, median ms, 3 × tree:0 → one combine fetch: gitlab-org/gitlab 716 → 249, linux stable 605 → 279, forgejo 649 → 341, gitlab-runner 663 → 221. K = 3 rejected: 145–566 KB and 0.8–1.6 s per fetch on those repos (K = 2: 20–55 KB). Steady-state requests (test/bench-rt.mjs, base 814922e → this): gitlab-runner readFile depth 1/2/3 4/5/6 → 3/3/4, listTree / and depth 1 3/4 → 2/2; forgejo depth 1/3/4 4/6/7 → 3/4/5; GitHub and Artifacts unchanged (GitHub 3 + n, Artifacts 3 at any depth). Learned per quirks record in a WeakMap (the Quirks JSON shape is unchanged), asked only once tree:0 is settled (so Artifacts and stock git never see it and #136's one refusal stands), first ask concurrent with the tree:0 commit fetch; a failed deep fetch is never used. SAFETY I42 |
208| `maxPackBytes` is enforced in `v2Step` (band 1 and band 2), not by each consumer; every object's zlib stream is capped at `zlibBound(size)` in `spStep` and `inflate` | the streamed blob paths had no body cap at all: endless empty stored blocks (5 bytes in, 0 out) were read at ~90 MiB/s until the request timer (shown: 793.6 MiB / 8 s). One choke point means a new consumer cannot forget the cap; the per-object bound (zlib's own `deflateBound`, since RFC 1951 has no ceiling) means a 10-byte blob costs ≤ 26 zlib bytes plus one chunk whatever `maxPackBytes` is, and a buffered pack's parse is linear in declared sizes |
209| Redirects never followed (`redirect: 'manual'`, 3xx → `http`); the credential scrub rides on the transport into every span and error | workerd 2026-10-23 forwards Authorization on a cross-origin redirect (measured); a server can echo the token in ERR/band 2/band 3/`ng` text. SAFETY I6b, I6c (issue #15) |
210| URL userinfo is sent as git sends it (`<token>@` → Basic `<token>:`, `:pass` → `:pass`, percent-decoded to bytes), not refused | measured on git 2.50 behind a Basic-auth gate: no prompt before the first credentialed request, the header byte-equal in every form; a library cannot prompt, so a refused one is `auth`. SAFETY I6d (issue #32) |
211| An explicit `auth` that sends nothing is `bad_request`; `{ user }` alone is refused, not `user:` | git's own credential API (a helper giving a username alone) asks for the password and sends nothing; a library cannot ask. `{ user, token: '' }` is the explicit `user:`. Explicit fields go out as UTF-8 bytes, as git sends a helper's. SAFETY I6e (issue #40) |
212| `planPackBatches`' `crossBatchRef` stays opt-in, and its cost when on is accepted (2026-11-11) | a delta whose base is in an earlier batch is written as REF_DELTA, a thin pack whether a host takes is the host's to say. With it off, planning is within 3% of main 8e3939d. With it on, each cut base's id is asked of `resolved` and checked: measured at 02374d4 against 8e3939d over 15 interleaved process rounds, 1.03 to 1.06 times on packs cut nowhere and 1.16 times (minimums 1.24) on an OFS pack in which nearly every push cuts a chain. README.md, planPackBatches |
213| A rebase walks its branch past `maxMergeDepth` in windows, and its refusal's request count is stated and asserted (2026-11-11) | the branch walk goes on from the cut frontier in windows of `maxMergeDepth` commits, one window held at a time as parent lists, so it holds no more than the merge-base search. A refusal costs 17 requests under `WORKER_LIMITS` (the merge-base search alone) and 26 under `DEFAULT_LIMITS`, both asserted in `test/rebase.test.js` on an `onto` history deeper than `maxMergeDepth`. README.md, rebaseBaseOf |
214| `limits.heldTimeoutMs` bounds how long a consumer may hold a chunk (default 4 x idle, its own `limit` name); SAFETY I1 is plain "No hang" again (Ivar, 2026-11-11, #273 option B; #281) | network idle is charged only while ply waits on the network, so before this nothing bounded a stalled consumer with default limits; the hold starts at the first clock read after the take (no clock read per chunk), so a cut is never early and at most min(idle, held) late |
215| `cachingFetch({ verifyBeforeStore: true })` stores only an answer `verifyFetchAnswer` accepted (wants present, objects hashed); without the flag the store path is main's eager put (2026-11-11, #270; #282) | #269 had kept the option's name while it did less (a want-less answer with a valid trailer was stored); `openBlob` has no bust path, so a poisoned entry must never be written |
216| `maxResolveWork` below `maxInflatedTotal` is refused at parse, `null` is refused like every key, and the resolver's ceiling is the parsed value (2026-11-11, #271; #280) | the old `Math.max` raised a caller's tight budget silently, and telemetry disagreed with the configuration |
217| A push whose body the transport never pulled is not a push: no settle reads, not counted, never `lastPush: 'refused'` (2026-11-11, #272; #283) | the command list never left the process, so no ref can have moved; the fact is carried in the type (`{ sent, result }`), not in the error code |
218| Regexes on remote text are linear-time, with no knownBad rows left in the regex inventory (2026-11-11, #274; #284) | the fetch filter-refusal matcher was quadratic on remote input (344 ms at 56 KB) |
220## 6. Verification — how to rerun everything
222Requirements: Node ≥ 22, git ≥ 2.43 on PATH, OpenSSH `ssh-keygen` for the signing suites, `workerd` (npm,
223pinned) for the Workers suite (skipped if absent).
225Before every push, run the fast gate. It is what CI's `lint + fuzz` job and the registry test reject first:
226types and the ratchets, lint, the space registry (gate a, SPACES.md freshness), and the built `dist` importing
227the way consumers import it (`scripts/dist-check.mjs` builds into a temporary directory and imports `ply` from
228there; your `dist/` is not rewritten, `npm run build` is what writes it). It needs no network. There are no git hooks: running it is the contributor's step,
229and `notify-ply-web` tells ply-web about a commit only after `ci` passed for that commit on main.
231```sh
232npm run check # typecheck + lint + registry test + dist check; run before every push
233```
235A ratchet baseline (`scripts/throw-baseline.json`, `scripts/absence-baseline.json`, `test/space/ratchet.json`,
236`test/durations.json`) moves the unsafe way only in a commit whose message has the line
237`Ratchet-Change: <file>: <reason>`. CI's `node scripts/ratchet-guard.mjs` compares each baseline with the base
238commit's and fails without that line; a squash merge must keep the line in its message, or main's own ci run fails.
239`npm run spaces -- --ratchet` only lowers `test/space/ratchet.json`. Adding a hole to it takes
240`npm run spaces -- --ratchet --init --reason "<why>"`, which prints that line; with no reason it exits 1, lists the
241holes it would add and writes nothing.
243The Git behaviour coverage matrix (`test/behaviours/matrix.ts`, rendered to `COVERAGE.md` by
244`npm run coverage-matrix -- --md`) counts server behaviours, not git commands: each has an area, the ply ops and
245tests that exercise it (grepped by `test/behaviours/matrix.test.js`, never asserted from memory), and a status of
246unsupported, implemented or verified. Verified carries evidence by kind (ply against git http-backend in this suite,
247ply, stock git or hand-built wire probes against Artifacts in a recorded run listed in `test/behaviours/runs.ts`);
248the types make a verified row without evidence impossible. A change that adds or retires a behaviour, a test, or a
249run updates the matrix and regenerates COVERAGE.md in the same commit.
251Differential probes turn matrix behaviours into scenarios (plans, types and the pure evaluator in `src/probe-*.ts`,
252exported as `ply/probes` so ply-web's workbench runs the same plans from a Worker; the runner in `test/probes/`): a pure typed plan per `<behaviour>#<variant>`
253(a seed of deterministic objects, so every target holds the same ids; pushes or concurrent pushes; an optional fault:
254hold the receive-pack request while another writer moves refs, or cut the server's answer) and the outcome git
255http-backend is expected to give. Each scenario runs by ply and by stock git against git http-backend configured to
256Artifacts' known policy (`test/probes/probes.test.js`, in CI) and, opt-in and private, against Artifacts
257(`npm run probes -- --artifacts --out <dir outside the repo>`). Stock git judges every run (ls-remote, mirror clone,
258fsck --strict, rev-list --parents, merge-base) and `src/probe-evaluate.ts` compares runs on meaning: outcome and
259error class, refs and HEAD, reachable objects, trees, ancestry, and whether a push a client called landed is in the
260refs. Expected differences live in `src/probe-allow.ts`, each with its reason; anything else is a finding,
261classified by triangulation (a server bug only when ply is right on the reference and stock git reproduces it on the
262server), and an unused allow entry fails the run. The repository keeps run ids and check names (runs.ts), never
263verdicts.
265```sh
266npm ci
267npm test # 113 deterministic tests + 16 fuzz stages (≈29 min)
268npm run coverage # line coverage of src/, seeds pinned (expects 100%)
269npm run fuzz:long # the long campaigns
270npm run test:push-full # the full push matrix (1,400 cells)
271```
273Individual oracles (each prints a repro on failure):
275| What | Command |
276|---|---|
277| Line merge vs `git merge-tree` | `node test/fuzz/linemerge.mjs --n 2000 --seed 1 [--runs\|--big] [--driver union\|binary] [--rediff myers\|skip\|undecided]` |
278| Diff hunks vs `git diff --histogram` | `node test/fuzz/diffscript.mjs --n 200 [--snake\|--huge]` |
279| `.gitattributes` vs `git check-attr` | `node test/fuzz/attributes.mjs --n 200 --seed 1` |
280| Wanted-tree filter model vs `rev-list` | `node test/fuzz/usergiven.mjs --n 60 --seed 500` |
281| Merge property (histories, renames, attributes) | `MERGE_SEED=11 MERGE_CASES=120 npx vitest run -t property test/merge.test.js` |
282| Rebase/cherry-pick property | `REBASE_SEED=1 REBASE_CASES=60 npx vitest run -t property test/rebase.test.js` |
283| Protocol product space | `node test/fuzz/protocol.mjs --n 200 --seed 32` |
284| CPU of inflate, parsePack and the push writer, old tree against new (interleaved, one process per case) | `npm run bench -- --runs 5 --tree <older checkout> --tree .` |
285| parseTree ns and bytes allocated per entry over every tree of a repository, old tree against new | `node test/bench-tree.mjs --runs 7 --corpus <repo>@<rev> --tree <older checkout> --tree .` |
286| Tree-entry name checks and parseTree against the pre-#207 code (test/oracle/) | `node --import ./test/ts-resolve.mjs test/fuzz/dotnames.mjs` (38.8 M names) and `test/fuzz/parsetree.mjs --n 1000000` |
288The oracle's version picks the matching mode automatically (`test/gitversion.js`). To judge against another git:
289`GIT_BIN=/path/to/git node test/fuzz/linemerge.mjs …` (and `diffscript`, `attributes`, `linemerge.test.js`).
291Building git master as the second oracle (what was used here, commit 47fdf220, "2.56"):
293```sh
294git clone --depth 1 https://github.com/git/git.git && cd git
295make -j NO_CURL=1 NO_OPENSSL=1 NO_EXPAT=1 NO_GETTEXT=1 NO_TCLTK=1 NO_PERL=1 NO_PYTHON=1 git
296```
298### Checks that only CI runs, and how to run them locally
300On 2026-11-11 every red head of review train 2 was red for a check that `npm run check` and a branch's own test
301files do not run: gate c claims (three rounds), surviving mutants (three rounds), git 2.56 against a laptop's git,
302the fail-site recorder, Linux `git init` defaults, and zlib output sizes. Each can be reproduced before a push.
304**Gate c claims for one space.** The registry (`test/space/registry.js`) gives each fail site in src/ to a space;
305gate c fails when a space's runners ran and none of them reached a site given to it. To judge one space, record over
306every runner file that space lists in the registry, then judge the records:
308```sh
309rm -rf .failsites
310PLY_FAILSITES=.failsites NODE_OPTIONS=--import=./test/space/failsite-hook.mjs npx vitest run test/space/describe-pack.test.js
311PLY_FAILSITES=.failsites PLY_FAILSITES_GATE=1 npx vitest run test/space/failsites.test.js
312```
314`npm test` records and judges the whole suite the same way. A space's runners are its `runners` list
315(`node -e "import('./test/space/registry.js').then(m => console.log(m.SPACES.find(s => s.name === 'describe-pack').runners))"`);
316record all of them, because a claim is judged as soon as any one of them is in the records. On a partial record like
317this one, only the `claims` test counts: `gate c: every fail site the registry gives a merged space is reached by
318that space` must pass. The `failsites` test fails, since most of src/ was not run, and that failure means nothing
319here; the Linux test is skipped off Linux (macOS hides cells, and CI decides). Reached means a recorded stack passed
320through the site's frame (`file:line:col`, `frameOf` in `test/space/holes.js`) while one of the space's runners ran.
321It does not mean a test read the error: `observed` reports reached sites whose `code` no test read, and is not
322judged. To tell whether a claim hole is new, record and judge a checkout of main the same way, with the same runner
323files, and compare the two `claims` lists.
325**The mutation gate.** Only the test files in `MUTATION_FILES` (`vitest.stryker.config.ts`) run under Stryker, so
326only they kill mutants. A test elsewhere that pins a line does not count: a mutant it would kill survives in CI. A
327test written to kill a mutant goes in one of those files, or its file is added to the list. A narrow run on a line
328range uses the repository's config (`stryker.conf.json`, which names `vitest.stryker.config.ts`):
330```sh
331npx stryker run --mutate 'src/format.ts:1-30' --concurrency 4
332```
334The dry run first runs every file in `MUTATION_FILES` (about 30 s here); the mutants of the range then run against the
335tests that cover them, and the clear-text report lists each survivor. Which survivors are fatal is decided in
336`scripts/mutate.mjs` only: a surviving or uncovered mutant in a parser function (the `PARSER` pattern there) or in a
337bounds file (`BOUNDS_FILES`: src/bytes, src/format, src/inflatestream, src/packstream) fails the build; others are
338reported. An equivalent mutant is allowed only as `// Stryker disable next-line <Mutator>: <reason>` on its line.
339`node scripts/mutate.mjs --check` judges an existing `reports/mutation/mutation.json` with CI's rules; on a narrow
340run it judges only the mutants that ran. CI runs the gate out of band (`mutate.yml`, 20 shards; `node scripts/mutate.mjs
341--budget` in job `lint + fuzz` fails when the heaviest planned shard passes 60% of the job limit), so a pull
342request's `ci` can be green while `mutate` is still red; merge after both.
344**CI's environment, not a laptop's.** CI runs Node 22 on ubuntu-latest, whose git is 2.56; a laptop often has
345another git (2.50 here) and Node 24. What differed:
347- Tests that read the git version (`test/gitversion.js`) take another branch on 2.56. Run them under a 2.56 build
348 (the build above): put its directory first on PATH, or `GIT_BIN=` for the oracles that take it.
349- git 2.56 started background maintenance after a push, and a repack renamed packs under a test's fsck. Every git
350 the suite starts is `test/git-shim/git` (first on PATH from `vitest.config.ts`), which turns auto-maintenance off;
351 a test that starts git some other way must go through it, or through `makeBareRepo` / `initRepo`, which set the
352 keys. `test/harness-maintenance.test.js` observes this.
353- Every CI test shard runs under the fail-site recorder (`PLY_FAILSITES` and the `NODE_OPTIONS` hook). It changes
354 stacks and the `code` accessor of ply's errors; a test that counts bytes, frames or properties of an error can pass
355 bare and fail there. Run a changed test once under the recorder (the first command above).
356- Linux `git init` writes no `core.ignorecase` or `core.precomposeunicode`; macOS writes both. A test that compares a
357 repository's config, or depends on name folding, must set them or not read them.
358- git's zlib and node's zlib can differ by a byte for the same objects, and which is smaller depends on the commit
359 timestamp. A test must not assume one pack is smaller than another; it measures both and sets its bound from the
360 larger (the `maxPackBytes` boundary in `test/push-repo-fs.test.js`).
361- `test/workflows.test.js` pins the workflow files' text. After editing a workflow, run it and the linters CI runs:
363```sh
364npx vitest run test/workflows.test.js
365actionlint
366uvx zizmor@1.31.1 --offline --persona regular .github/workflows # CI's version; config in .github/zizmor.yml
367```
369A pass is the test file green, actionlint silent with exit 0, and zizmor's "No findings to report".
371**Reviewing a change.** Run every test file that imports a changed src module, not only the test files the change
372touched. A closure check that ran only a branch's own tests passed while about 25 tests elsewhere failed (a change
373to how `spanOn` treats a tracer). To list them: `git diff --name-only main... -- src` for the modules, then
374`git grep -l` for each module's import path under test/. When a train head is red, bisect along its merges.
376**Benchmarking a perf bar.** On a loaded machine, a number from one process run is noise. Interleave processes of
377main, the candidate (head) and a second copy of main (A/A), one sample per process, many rounds. The verdict is
378(head/main) / (aa/main): a candidate is faster only by more than the A/A arm moved. Sample memory peaks every 2 ms;
379a coarser interval misses them. Use realistic input sizes: a win measured on 1 to 8 byte inputs was an artifact
380that did not hold at real sizes.
382## 7. Numbers (2026-10-20)
384- Bundles (esbuild, minified): edit 42.6 KB (16.5 KB gz); merge 63.9 KB; full 116 KB.
385- Line merge: 0 wrong across >40,000 random merges against 2.43 and 2.56; union 5,280 byte-identical; diff hunks identical
386 on >1,400 diffs incl. 947 Myers heuristic splits and 160+ mxcost cutoffs.
387- Attributes: >40,000 paths equal to `check-attr` on both versions; 380/380 git wildmatch vectors.
388- Merge property runs: 0 wrong across seeds 11–15 with merge-commit and criss-cross histories and random attributes.
389- Coverage: 100% of 2,461 lines.
391## 8. Open gaps and next steps
393In order of value:
3951. **Real keys**: a GitHub push run (receive-pack policy layer, protected branches, the exact `ng` wording the
396 `cas_conflict` classifier relies on) and an observed run on the Workers edge (CPU, subrequest and memory limits).
3972. **Rename detection** (merge-ort's): today every rename shape is a conflict — the largest source of conservative
398 conflicts on real refactors.
3993. **Virtual merge base** for criss-cross histories (refused today).
4004. **Conflict output**: we refuse rather than produce markers/partial results; no resolution flow.
4015. Server-side configuration ply cannot see: `info/attributes`, custom merge drivers, `merge.default`.
4026. Rebase niceties: patch-id skipping of already-upstream commits; merges in the chain (`not_linear` today).
4037. Protocol/transport limits: SHA-256 repositories; servers without protocol v2; no SSH transport (removed 2026-10-31;
404 history on branch archive/ssh).
4058. Features not built: annotated tags, multi-file edits on top of the parent tree (pushTree replaces the whole tree),
406 LFS beyond the guard, merges of files over 8 MiB, a `ply push <dir>` CLI and `.gitignore` reading for `pushDir`;
407 for checkoutTree: submodule recursion, `.gitattributes` (eol/filters) and LFS smudging, sparse fetching of only `path`'s
408 trees, and checkout into a non-empty directory.
410## 9. What is in this export
412```
413ply-export/
414 ply/ the project (no node_modules, no fuzz-out traces) — run `npm ci`
415 README.md user guide
416 SAFETY.md invariants, findings, run record, what is not proven
417 CONTEXT.md this file
418 src/ test/ package.json package-lock.json
419 process/
420 README.md index of the logs and experiments below
421 logs/ the final npm test run, the coverage report, the 20k line-merge campaign
422 experiments/ the one-off scripts behind the findings (criss-cross, trap, chain limit, …)
423```
Diagnostics5 git calls · 93 ms
CallStatusSizeTime093 ms
discover200191 B2 ms
git.upload-pack discover
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=2
git.response.bytes=191
git.server.agent=git/github-9922ec520325-Linux
git.server.object_format=sha1
git.server.caps=agent ls-refs fetch server-option object-format
git.server.fetch=shallow wait-for-done filter
git.protocol.version=2
git.pkt.frames=9
git.pkt.data=7
git.pkt.flush=2
git.pkt.delim=0
git.pkt.bytes=191
git.pkt.payload.max=36
ls-refs20012 KiB · hit3 ms
git.upload-pack ls-refs
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=ls-refs
git.upstream.dur_ms=3
git.response.bytes=11990
git.response.cache=HIT
git.response.duration_ms=3
git.protocol.version=2
git.pkt.frames=152
git.pkt.data=151
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=11990
git.pkt.payload.max=122
git.ls_refs.count=151
git.ls_refs.head=refs/heads/main
fetchtree:0 · deepen 102004.2 KiB · hit2 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=2
git.response.bytes=4347
git.response.cache=HIT
git.response.duration_ms=2
git.fetch.filter=tree:0
git.fetch.wants=19912563f540228593df0c489082e5e2508e21e4
git.fetch.haves.count=0
git.fetch.deepen=10
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=7
git.pkt.data=5
git.pkt.flush=1
git.pkt.delim=1
git.pkt.bytes=4347
git.pkt.payload.max=4247
git.sideband.band1.bytes=4247
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=10
git.pack.bytes=4247
git.pack.ofs_delta=0
git.pack.ref_delta=0
git.pack.chain.max=0
git.pack.trailer=a6caf8e129bea82c9bae4cbc7c130c9205add400
git.pack.full=10
Open pack map
fetchtree:0200919 B · hit1 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=1
git.response.bytes=919
git.response.cache=HIT
git.response.duration_ms=1
git.fetch.filter=tree:0
git.fetch.wants=800d30ba8923860bfc460ca884b7f9ba22083588
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=4
git.pkt.data=3
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=919
git.pkt.payload.max=892
git.sideband.band1.bytes=892
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0
git.pack.objects=1
git.pack.bytes=892
git.pack.trailer=5323b9266eea5ded637285d1c5320699f12be76e
Open pack map
fetch20022 KiB87 ms
git.upload-pack fetch
http.response.status_code=200
git.upstream.cf_ray=-
git.upstream.phase=fetch
git.upstream.dur_ms=87
git.response.bytes=22663
git.response.cache=MISS
git.response.duration_ms=87
git.fetch.wants=52c225f23b9b06d2b86c2c1e753bdb919c9575ab
git.fetch.haves.count=0
git.fetch.deepen=0
git.fetch.thin_pack=false
git.fetch.ofs_delta=true
git.protocol.version=2
git.pkt.frames=5
git.pkt.data=4
git.pkt.flush=1
git.pkt.delim=0
git.pkt.bytes=22663
git.pkt.payload.max=14440
git.sideband.band1.bytes=22631
git.sideband.band2.bytes=0
git.sideband.band3.bytes=0

Each row is one HTTP round trip ply made to the git server for this page, on one timeline. discover asks what the server speaks, ls-refs lists branch tips, fetch brings objects by id. A hit was served by Workers Cache. Tap a row for its wire details.

Remote
https://github.com/ivarvong/ply
Ref
refs/heads/main
Commit
19912563f540228593df0c489082e5e2508e21e4
cf-ray
a5e0e9695c6aa140
Request
c16b2cfe…
Protocol
v2 · sha1
Agent
git/github-9922ec520325-Linux
Filter
advertised
Fetch
shallow wait-for-done filter
Capabilities
agent ls-refs fetch server-option object-format