Document physical Metal TP QA preflight and native batch coverage
Check per-boot RDMA addressing and wired residency limits, match prefill chunk overrides on both ranks, and account for the extra sessions used by numerical oracles. Require at least three Metal TP rows when checking the native V4.1 path; two rows intentionally use ordered decoding.
This commit is contained in:
1 parent
6687c6f90b
commit
0aaea5a238
1 file changed
+16
-2
+16
-2
@@ -335,6 +335,16 @@ or backend fallback selection changes.
|
||||
the shared expert, but still runs attention, routed experts, shared down, and
|
||||
the output head per session. Treat flat aggregate scaling as unfinished
|
||||
implementation work, not evidence that Metal cannot benefit from batching.
|
||||
- Before physical Metal TP, check both hosts against the link and memory setup
|
||||
in [the distributed guide](docs/DISTRIBUTED.md). Reboots can reset
|
||||
member-interface IPv4 aliases and `iogpu.wired_limit_mb`. An active port is not
|
||||
enough, and a shard above the effective GPU residency limit may page heavily.
|
||||
Preserve OS headroom; do not bypass the memory guard to make a test fit.
|
||||
Match explicit `--prefill-chunk` overrides on both ranks. A different chunk
|
||||
schedule can fail during prefill synchronization rather than at the handshake.
|
||||
The batch oracle needs extra sessions for mixed prefill and serial controls.
|
||||
Use an admitted context for its short-prompt checks, then test long contexts
|
||||
separately rather than skipping the mixed checks.
|
||||
- On `mac-m5max-it` and `mac-m5max-us`, run the same oracle in physical TP mode
|
||||
over explicit `tcp` and `rdma` transports. Set `DS4_TEST_TP_MODE=leader` on
|
||||
the leader and `DS4_TEST_TP_MODE=worker DS4_TEST_TP_LEADER_HOST=HOST` on the
|
||||
@@ -1519,8 +1529,12 @@ the long sparse-boundary tests; neither substitutes for the other.
|
||||
prompts and row order, checks complete target logits exactly, tests invalid
|
||||
batches without advancing state, and resumes after a mixed prefill/decode call.
|
||||
Isolation does not establish equivalence to serial inference: separately run
|
||||
`score_official --session-batch N` on the short and long manifests. Measure
|
||||
aggregate throughput without Metal validation, including two-session cases.
|
||||
`score_official --session-batch N` on the short and long manifests. Test
|
||||
physical Metal TP native batches with at least three rows: two rows use
|
||||
ordered execution by design. Confirm dispatch in the log instead of assuming
|
||||
that `--session-batch` selects the native path.
|
||||
Measure aggregate throughput without Metal validation, including two-session
|
||||
cases.
|
||||
Require the `native_ds41=1` trace for admitted native shapes, including mixed
|
||||
batches with nonzero `prefill_rows`; a passing ordered fallback is not evidence
|
||||
for the native path. Image-bearing sessions currently use that fallback.
|
||||
|
||||
Reference in new issue
Block a user