Giorgio Oppo
575369b19d
Archive the investigation of PR #1056 comment 5963229415 comparing main0aaea5awith02313128on M4 Max/Q2 resident. Trace the device-independent prefill policy introduced by2d6a207b: small F16/Q8 prompt batches change operand precision, while GDN alpha/beta F32 projections stop using split-K. Distinguish this policy from the numerically neutral M2-M4 tuning gate,89986c3cattention partitions and the secondary HC arithmetic candidate. Record ten full-vocabulary captures and four 64-step greedy comparisons from one frozen HEAD snapshot on M1 Max/Q4/SSD, with inherited DS4 flags removed. Restoring only F32 split-K changes the 575-token frontier by 0.41702342; restoring the complete dense policy produces identical logits on that fixture. The 29/575 repeated HEAD controls are byte-identical. The 5942-token tail-only split ablation changes logits by 0.33132219. With ordinary T1 decode policy held constant, the dense-policy ablation changes the first generated token at step 59 for Rome29 and step 20 for book575. Preserve raw logits, top-20 logprobs, logs, token-ID fixtures, commands, source hashes and independent provenance/numerical audits. These fixtures do not reproduce the tester's exact M4/Q2 prompts or attribute every component of the reported deltas; they are not quality or performance evaluations. Provide a preparation script that recreates 103 tracked source files from0231312820, applies the isolated diagnostic patch and verifies all source hashes. Add fresh snapshot/output arguments to both runners and document serial reproduction using the local Q4 GGUF. Keep generated snapshot copies, executables and object files outside Git. Validation: - Independent recalculation of all ten 248320-logit dumps and greedy common prefixes, selected token IDs and bytes passed. - Preparation in a fresh temporary directory reproduced all 103 source hashes and applied only the reviewed diagnostic patch. - Python AST/JSON parsing and both runner help commands passed. - git diff --cached --check passed. This commit archives research only; pending production optimization experiments remain outside its scope. No production dispatch is changed.
44 lines
2.8 KiB
Python
44 lines
2.8 KiB
Python
from pathlib import Path
|
|
import argparse
|
|
import hashlib,json,os,subprocess,time
|
|
lane=Path(__file__).resolve().parent
|
|
repo=lane.parents[1]
|
|
parser = argparse.ArgumentParser(description="Repeat 64-step greedy comparisons after the prefill ablation.")
|
|
parser.add_argument("--snapshot", default="head", help="Snapshot directory, relative to this archive or absolute")
|
|
parser.add_argument("--output", default="greedy-ab", help="Fresh output directory, relative to this archive or absolute")
|
|
args = parser.parse_args()
|
|
head = (lane / args.snapshot).resolve()
|
|
out = (lane / args.output).resolve()
|
|
out.mkdir(exist_ok=False)
|
|
env={k:v for k,v in os.environ.items() if not k.startswith("DS4_")}
|
|
report={"scope":"controlled HEAD dense policy ablation, M1 Max Q4 SSD, MTP off", "runs":[],"comparisons":[]}
|
|
ref={}
|
|
for name in ("rome29","book575"):
|
|
for arm in ("head","base_dense"):
|
|
run_env=env|({"DS4_DIAG_BASE_PREFILL_DENSE":"1"} if arm=="base_dense" else {})
|
|
target=out/f"{name}-{arm}.json"
|
|
cmd=[str(head/"ds4"),"-m",str(repo/"gguf/Qwen3.8-Flash-Next-Q4.gguf"),"--metal","--ssd-streaming","--ctx","16384","--prefill-chunk","2048","--nothink","--temp","0","-n","64","--prompt-file",str(lane/"fixtures"/(name+".txt")),"--dump-logprobs",str(target)]
|
|
print("START",name,arm,flush=True)
|
|
begin=time.monotonic()
|
|
with (out/f"{name}-{arm}.log").open("wb") as log:
|
|
rc=subprocess.run(cmd,cwd=head,env=run_env,stdout=log,stderr=subprocess.STDOUT).returncode
|
|
assert rc==0
|
|
data=json.loads(target.read_text())
|
|
ids=[step["selected"]["id"] for step in data["steps"]]
|
|
report["runs"].append({"name":name,"arm":arm,"seconds":time.monotonic()-begin,"command":cmd,"env":{k:v for k,v in run_env.items() if k.startswith("DS4_")},"steps":len(ids),"ids":ids,"json_sha256":hashlib.sha256(target.read_bytes()).hexdigest()})
|
|
if arm=="head":ref[name]=data
|
|
else:
|
|
original=ref[name];a=[x["selected"]["id"] for x in original["steps"]]
|
|
divergence=next((i for i,(x,y) in enumerate(zip(a,ids)) if x!=y),None)
|
|
if divergence is None and len(a)!=len(ids):divergence=min(len(a),len(ids))
|
|
c={"name":name,"same_tokens":a==ids,"first_different_step_zero_based":divergence,"steps_head":len(a),"steps_base_dense":len(ids)}
|
|
if divergence is not None:
|
|
c["selected_head"]=original["steps"][divergence]["selected"]
|
|
c["selected_base_dense"]=data["steps"][divergence]["selected"]
|
|
report["comparisons"].append(c)
|
|
print("COMPARE",json.dumps(c),flush=True)
|
|
(out/"results.json").write_text(json.dumps(report,indent=2)+"\n")
|
|
print("DONE",name,arm,round(time.monotonic()-begin,2),flush=True)
|
|
report["status"]="COMPLETE"
|
|
(out/"results.json").write_text(json.dumps(report,indent=2)+"\n")
|