rapidocr/README.md

# Pure C++ PP-OCRv6 inference

This is a focused C++ port of the PP-OCRv6 detector/recognizer path from
`D:\workprj\aicoder\corelib\ocr`.  It has no dependency on ONNX Runtime,
Paddle Inference, OpenCV, or Python.

The library is also a C and C++ API (`include/ppocr/ppocr.hpp`,
`include/ppocr/ppocr.h`) with static and shared builds, CMake package
exports, and a Python package whose `RapidOCR` class follows the interface
on `main`. Usage, the stable C ABI, and the CPU dispatch rules are in
[`docs/API.md`](docs/API.md). Measured AVX2 / AVX-512 / NEON notes are in
[`docs/PERFORMANCE.md`](docs/PERFORMANCE.md).

The embedded executor is intentionally not a general ONNX Runtime.  It only
implements the operator set exercised by the official PP-OCRv6 tiny/small/
medium ONNX model bundle: convolutions (including transpose), pooling,
elementwise math, normalisation, resize, tensor-shape operators, attention
matmul, and softmax.

> **Current GPU-only qualification (2026-08-21).** The strict Vulkan
> GPU-only OCR path runs RGB preprocessing, the complete neural graph, DB
> postprocessing, and CTC Top-1 on the GPU; it has no CPU neural-operator
> fallback. On the local AMD Radeon 780M / LLPC stack, exact CPU/GPU decoded
> text and geometry are currently verified for PP-OCRv6 **tiny** and **small**.
> GPU recognizer crops
> now retain their exact PP-OCR aspect-ratio width rather than adding a
> 64-column scheduling suffix, fixing short-crop CTC parity. The **medium**
> detector at production `1x3x160x704` remains unqualified and fails closed;
> no result is substituted from CPU. Nearby `1x3x160x640` and `1x3x160x768`
> shapes remain CPU/GPU parity-qualified. Historical GPU claims below do not
> widen this current qualification.

### 2026-08-22 strict GPU-only public-boundary recheck

The public `gpu_only` mode remains a strict Vulkan-only execution path:
packed RGB upload and resize, detector and recognizer graphs, DB box
postprocessing, and CTC Top-1 execute on the device. Failed Vulkan execution
does not retry neural inference on CPU.

This recheck also carries the Vulkan queue/fence status through every public
GPU-only OCR stage (RGB front end, detector, DB postprocess, recognizer and
CTC). A device reset is therefore reported as `VK_ERROR_DEVICE_LOST`, not
mislabelled as an unsupported ONNX operator. The only retry allowed by the
GPU-only page batch scheduler is an N=1 Vulkan retry after a rejected N-batch;
it never invokes the CPU executor. On the local Radeon 780M / LLPC adapter,
production medium-detector `1x3x160x704` still fails closed with that result.
Changing command-segment size, forcing queue-idle, using a fresh command
buffer, using the isolated terminal-depthwise shader, or using its
generic/sliced variant did not make that shape stable. This is a driver
compatibility limit, not a CPU-fallback opportunity.

### 2026-08-22 GPU-only deployment coverage

`gpu_only` is a complete execution mode, not a request to accelerate only
selected layers: packed RGB upload/resize, detector, DB postprocess,
recognizer, CTC Top-1, and the real NCHW page/crop batch paths stay on one
Vulkan queue.  Only final compact OCR records and UTF-8 construction reach
the host.  A rejected N-batch is retried as strict Vulkan N=1; it is never
redirected to the CPU graph.

The Vulkan loader is dynamically opened at runtime, so applications do not
link to Vulkan or ONNX Runtime.  The build now embeds shaders with the
vendored Windows compiler when available, otherwise with a host
`glslangValidator`/`glslang`; the same executor is therefore buildable on
Windows x86/x64, Linux x86/x64, and Linux ARM/AArch64 when Vulkan headers and
a shader compiler are supplied.  If the loader, compute queue, shader build,
or a required graph node is unavailable, constructor/runtime validation fails
closed rather than selecting CPU inference.

After this portability change, the local Radeon 780M validation was rerun
with the rebuilt `build-gpu-only-ep` binaries:

| Check | Result |
| --- | --- |
| tiny GPU-only OCR, `en` + `mixed` + low contrast, 2 repeats | passed; 8 records; max confidence delta `2.50e-5` |
| small GPU-only OCR, `en` + `mixed` | passed; 5 records; max confidence delta `1.0e-6` |
| tiny two-page GPU batch, `en` x2, 1 warm-up + 5 runs | serial `93.109 ms`, batch `84.491 ms`, `1.102x`; text/geometry equivalent |
| tiny GPU memory ladder, 16px through 1920px pages then back to small | all requests completed; post-large small/en/mixed returned to the warmed `596,709,376 B` private-byte baseline |

The last memory sweep later rose by at most `9,392,128 B` after a noisy page
and finished `7,172,096 B` above that process allocator baseline.  This is a
bounded host allocator effect observed after the request; it is not reported
as GPU arena growth and does not relax the device-only inference contract.

### 2026-08-22 expanded GPU-only corpus recheck

The correct PP-OCRv6 common dictionary (`dict_ppocrv6.txt`, vocabulary
`18710`) was used for small/medium rather than the tiny dictionary.  The
latest strict GPU-only corpus covers low contrast, very wide/tall source
images, and a 1920×1080 screenshot in addition to short English/mixed text:

| Model | Images | Result |
| --- | --- | --- |
| tiny | `en`, `mixed`, low contrast, `3000×80`, `80×3000`, screenshot; 2 repeats | passed; 63 records; exact decoded text/geometry; max confidence delta `1.1662e-3` |
| small | same six images | passed; 63 records; exact decoded text/geometry; max confidence delta `2.265e-4` |
| medium | complete OCR at production detector shape | unavailable on this Radeon 780M/LLPC driver: strict `VK_ERROR_DEVICE_LOST`; no CPU substitution |

The independent neural-graph checks remain passed for tiny/small
`1×3×160×704` and medium `1×3×160×640`, with maximum CPU/GPU errors
`5.63276e-7`, `3.05939e-7`, and `7.56991e-8` respectively. Small-model memory
through the same small-to-large ladder returned to its warmed baseline after
the final screenshot (`1,020,276,736 B` private bytes); a low-contrast request
briefly added `14,577,664 B` in the process allocator and the final screenshot
was below baseline. `ui_dense_1920x1080` is deliberately excluded from the
qualified OCR corpus: it currently has one CPU/GPU decoded-result mismatch.
That failure is retained as a regression target rather than being masked by a
fallback or presented as a pass.

The rebuilt `build-gpu-only-ep` binaries passed the currently qualified
coverage. The graph timing measurements below use two warmed runs on the same
machine and include the current host/device graph boundary:

| Check | Result |
| --- | --- |
| tiny detector graph, `1x3x160x704` | passed; max error `5.63276e-7`; CPU/GPU `10.590 / 45.256 ms` |
| small detector graph, `1x3x160x704` | passed; max error `3.05939e-7`; CPU/GPU `18.676 / 66.362 ms` |
| medium detector graph, `1x3x160x640` | passed; max error `7.56991e-8`; CPU/GPU `80.312 / 272.255 ms` |
| GPU-only OCR tiny, `en` + `mixed` | passed; 5 records; max confidence delta `2.9e-6` |
| GPU-only OCR small, `en` | passed; 2 records; max confidence delta `8e-7` |
| true two-page GPU batch, tiny `en` x2 | serial-equivalent; `93.177 -> 90.993 ms` (`1.024x`) |
| true two-page GPU batch, small `en` x2 | serial-equivalent; `239.759 -> 231.595 ms` (`1.035x`) |

The medium detector memory ladder
`64x128 -> 96x256 -> 128x512 -> 160x640 -> 64x128` also passed. At every
phase the live allocation returned to the `61,946,548`-byte immutable
constants baseline; reusable capacity reached `201,457,664` bytes and is
intentionally retained for later requests. These results validate qualified
GPU-only execution and recovery behavior, but do not claim that this UMA GPU
is faster than the AVX-512 CPU or an ORT GPU provider.

The public GPU-only memory ladder also covered, in increasing source-size
order, `tiny_16x16`, `en`, `mixed`, `noisy_lowcontrast`, `wide_strip_3000x80`,
`tall_strip_80x3000`, `screenshot_1920x1080`, and `ui_dense_1920x1080`.
After the final large image, processing the next small image retained no extra
private bytes above the warmed large-page baseline. A later noisy page can
make the process allocator grow by about 12 MB for compact host result/control
storage; that allocation stayed bounded through the remaining large pages.
GPU replay command buffers and their pinned slots are now discarded whenever
the model uses the bounded detector/recognizer hand-off policy, so old replay
shapes cannot accumulate as device allocations.

### 2026-08-22 hybrid Vulkan admission correction

Hybrid terminal CTC GEMM now follows the documented policy exactly: for the
same model shape, CPU worker budget, Vulkan adapter, and Vulkan runtime
generation, it is admitted when a complete upload/compute/download boundary
is **no slower than CPU**. The older `GPU <= CPU / 2` rule could reject an
equal-speed GPU path merely because multiple recognizer crops might execute
concurrently on CPU. Queue serialization is instead controlled at the OCR
scheduler boundary. The admission cache is now also tagged with the Vulkan
runtime generation, so a device-loss rebuild always re-measures rather than
reusing an answer from the old device.

On this Radeon 780M host, tiny `en` CTC terminal shapes measured `1.361 /`
`0.718 ms` and `1.846 / 1.281 ms` for GPU/CPU, so they correctly remain CPU
on this adapter. This is expected: the implementation enables Vulkan only on
actual no-slower results, and does not manufacture a GPU claim from a slower
boundary. The hybrid batch regression (`tiny_16x16`, `en`, `mixed`, low
contrast; 2-page scheduling) remained text/geometry equivalent and measured
`67.358 -> 63.918 ms` (`1.054x`).

### 2026-08-21 GPU-only requalification

#### Latest local recheck

The strict Vulkan build was rebuilt after removing an unproven automatic
generic-tail experiment. The standard terminal scalar depthwise selection is
restored for qualified shapes. The following tests were rerun on AMD Radeon
780M / LLPC; all use the pure C++ implementation without ONNX Runtime:

| Check | Result |
| --- | --- |
| tiny detector graph, `1x3x160x704` | passed, max error `5.63276e-7` |
| medium detector graph, `1x3x160x640` | passed, max error `7.56991e-8` |
| medium detector graph, `1x3x160x768` | passed, max error `3.25672e-8` |
| medium recognizer graph, `1x3x48x320` | passed, max error `2.66433e-5` |
| GPU-only OCR tiny, `en` + `mixed` | passed; 5 records; max confidence delta `2.9e-6` |
| GPU-only OCR small, `en` | passed; 2 records; max confidence delta `8e-7` |
| tiny graph memory ladder, `64x128 -> 96x256 -> 128x512 -> 160x704 -> 64x128` | passed; live device bytes always returned to `1,713,924` B |

The production medium detector at `1x3x160x704` still returns
`VK_ERROR_DEVICE_LOST` on this driver after the terminal `896x5x22` depthwise
stage. With the normal 20-node segmentation the failure is observed on the
segment ending at that stage; a diagnostic command-stream grouping can submit
that tail and subsequently fails in the FPN head. This identifies a
Radeon 780M/LLPC Vulkan command-stream stability issue rather than a usable
CPU fallback point. It remains deliberately fail-closed: no CPU neural or
postprocess substitute is used. This blocks a claim of complete all-model
GPU-only qualification on the local adapter.

The end-to-end GPU path was rebuilt from the conservative source and retested
on Radeon 780M/LLPC. A GPU crop may use a dynamic width; it is no longer
rounded to 64 columns, because padded input columns produce extra valid
CNN/CTC time rows and can alter a short crop's text or confidence.

| GPU-only OCR pair | Corpus | Result |
| --- | --- | --- |
| tiny | `tiny_16x16`, `en`, `noisy_lowcontrast`, `wide_strip_3000x80` | 7 results; exact text/geometry; max confidence error `2.50e-5` |
| tiny replay | `en` repeated 5 times on one live OCR instance | 10 results; exact text/geometry; max confidence error `4e-7` |
| small | `en`, `noisy_lowcontrast` | 5 results; exact text/geometry; max confidence error `1.3e-6` |

The strict graph sweep (three runs except medium detector, two) also passed:
tiny detector `9.367 / 28.870 ms` CPU/Vulkan, tiny recognizer `5.224 / 23.189
ms`, small detector `17.976 / 48.075 ms`, small recognizer `13.320 / 62.323
ms`, medium recognizer `30.569 / 123.212 ms`, and the qualified medium
detector shape `1x3x128x704` `78.976 / 234.478 ms`. These are full graph
latencies on this UMA GPU, not an assertion that it beats the AVX-512 CPU.
Maximum graph-output errors were `5.63e-7`, `2.18e-5`, `3.06e-7`, `3.78e-4`,
`2.66e-5`, and `6.71e-9` respectively.

GPU arena recovery was also rerun after the correction. In every small-to-
large-to-small phase, dynamic `activation_bytes=0` after completion:

| Graph ladder | Immutable live baseline | Final reusable capacity |
| --- | ---: | ---: |
| tiny detector through `1x3x160x704` | 1,713,924 B | 47,513,600 B |
| small detector through `1x3x160x704` | 9,813,844 B | 75,235,328 B |
| medium recognizer through `2x3x48x320` | 76,456,140 B | 174,850,048 B |

### 2026-08-21 verification update

The strict GPU graph replay policy was corrected for public host-input
`Run()` calls.  Replay remains enabled for device-input and in-graph RGB
execution, while host-input calls now release their dynamic activation slots
when the submission completes.  This is an arena-lifetime change only: every
neural operator remains a Vulkan dispatch and an unsupported or failed device
operation still fails closed.

The reproducible tiny-detector ladder
`1x3x64x128 -> 1x3x96x256 -> 1x3x128x512 -> 1x3x160x704 -> 1x3x64x128`
now passes `ppocr_gpu_graph_mem_soak`.  Its immutable live baseline is
**1,713,924 bytes**; after every sample, including the largest one, live
bytes return exactly to that baseline.  Reusable arena capacity grows to
**47,513,600 bytes** at the largest input and intentionally stays available
for the final small input, so capacity is not misreported as a live-memory
leak.

The same live-memory invariant also passed for the qualified medium-detector
ladder ending at `1x3x160x640`: its **61,946,548-byte** immutable baseline
was recovered after each sample, with a stable **216,072,192-byte** reusable
high-water capacity.

The medium detector was rechecked at the production `1x3x160x704` input.
On this Radeon 780M/LLPC driver it still fails closed with
`VK_ERROR_DEVICE_LOST` in the terminal `896x5x22` depthwise dispatch. The
dedicated scalar shader is now selected automatically for that exact Vulkan
operator (use `PPOCR_GPU_TAIL_GENERIC=1` only for diagnostics); it also fails
closed on this production shape, as did device-local and host-cached arena
variants. The nearby qualified `1x3x160x640` and `1x3x160x768` shapes pass
CPU/Vulkan output comparison, which further confirms this is a driver/shape
interaction rather than a CPU fallback.

Hybrid RGB front ends are now admission-driven by default: detector RGB+stem
and recognizer RGB+Conv.0 run only after their complete upload/dispatch/
readback boundary is both numerically validated and no slower than the native
SIMD path. `PPOCR_DISABLE_VULKAN_RGB_STEM` and
`PPOCR_DISABLE_VULKAN_REC_RGB_CONV` disable those probes for diagnostics; no
enable-only switch is required.

The default strict-graph command segment is now **20 nodes** (override with
`PPOCR_GPU_ONLY_SEGMENT_NODES`).  A fresh three-run Vulkan sweep selected that
setting for the Radeon 780M: detector graph means were tiny `30.201 ms`,
small `46.590 ms`, and qualified medium `239.388 ms`; recognizer means at
`1x3x48x320` were tiny `23.169 ms`, small `60.031 ms`, and medium
`123.182 ms`.  CPU/Vulkan maximum output errors remained respectively
`5.63e-7`, `3.06e-7`, `6.71e-9`, `2.18e-5`, `3.78e-4`, and `2.66e-5`.
This is a segment-lifetime optimization, not an assertion that the local UMA
GPU is faster than the AVX-512 CPU.

Memory recovery was also exercised across detector resolution and recognizer
width/batch diversity.  Small detector `64x128 -> 96x256 -> 128x512 ->
160x704 -> 64x128` always returned to its **9,813,844-byte** constants
baseline.  The medium recognizer ladder `1x3x48x64 -> 1x3x48x160 ->
1x3x48x320 -> 2x3x48x320 -> 1x3x48x64` always returned to its
**76,456,140-byte** constants baseline.  Capacity is intentionally retained
as reusable GPU storage; no dynamic activation remains live after a sample.

The CPU batch executor now supports real same-shape detector NCHW batches
through both fused `MaxPool -> Concat -> Conv` and `Concat -> Conv` nodes.
Those nodes now advance each source by its own NCHW channel-plane stride, so
unequal concat branches cannot cross-contaminate pages. The varied-size
four-image detector-batch regression passes for tiny/small/medium: decoded
text and geometry are exact, while FP32 confidence is checked against its
documented bounded tolerance. `rec_batch_size` now defaults to `1` because
this PP-OCRv6 transformer export can change CTC results for N>1 recognition
crops; deployments may still opt in and qualify it independently. GPU-only
retains its separate device-resident crop batching path.

The full small-to-large-to-small memory ladder (19 images, including
`5120x2880`, `7680x4320`, wide and tall aspect ratios) also passed for every
CPU model.  Per-resolution private-byte and working-set evidence is retained
in `build-gpu-only-final/memory_ladder_{tiny,small,medium}_cpu_20260821.json`.
In each run both counters fell below the post-8K high-water during the
recovery tail; final-small private memory was within **-0.05% / +0.33% /
+1.06%** of the warmed tiny/small/medium baselines respectively.

The current five-image CPU comparison is recorded in
`build-gpu-only-final/cpu_diverse_benchmark_20260821.json`. On this local
AVX-512 host (16 threads, two warmups, eight runs per case), native C++ beat
the installed ONNX Runtime 1.28.0 CPU provider on every case:

| PP-OCRv6 model | weighted latency improvement | weakest case |
| --- | ---: | ---: |
| tiny | 73.54% | 63.33% |
| small | 46.16% | 36.22% |
| medium | 9.22% | 1.45% |

This is a local CPU-only result. The installed ORT package lists
`AzureExecutionProvider` and `CPUExecutionProvider`; its observed variance,
especially on small inputs, means these figures are not a portable GPU or
architecture-wide claim.

## CPU acceleration and architecture support

The inference path remains dependency-free and portable across x86/x64 and
ARM/AArch64. Hot contiguous elementwise paths (Add/Sub/Mul/Div, ReLU,
HardSwish, and affine scale/shift),
pointwise `1x1` convolutions, ordinary/depthwise `3x3`/`5x5` convolutions,
spatial mean reduction, fused BatchNorm+GELU, and attention `MatMul` use
fused vector kernels:

- x86/x64: AVX-512F/FMA when supported by the running CPU and OS; otherwise
  AVX2/FMA; otherwise scalar code. Advanced ISA code is compiled into separate
  translation units and selected only after runtime CPUID/OS-state checks.
- ARM/AArch64: NEON when the compiler/target exposes it, with a scalar
  fallback otherwise.
- Larger GEMMs are divided over CPU cores; small GEMMs stay single-threaded to
  avoid thread-launch overhead. Direct convolutions use a persistent worker set
  (rather than creating threads per operator), defaulting to at most 16 workers;
  `PPOCR_BENCH_THREADS` is an explicit override for deployment tuning and A/B
  tests.
- PPOCR_DISABLE_AVX512=1 is an optional per-process A/B switch for x86
  deployments. It keeps all runtime safety checks and selects AVX2/FMA when a
  particular AVX-512 CPU or workload performs better at its non-AVX-512 clock.
- `ppocr_kernel_smoke` verifies paired/odd GEMM rows, scalar/vector column
  tails, optional bias, and a recognizer-sized projection. Run it normally and
  with `PPOCR_DISABLE_AVX512=1` to cover the dynamic AVX-512 and AVX2 paths.
- PPOCR_DISABLE_AVX512_POINTWISE8=1 is a narrower A/B switch for the AVX-512
  eight-output `1x1` register tile. It retains AVX-512 and its four-output
  pointwise kernel, so deployments can isolate this tile's cache/register
  trade-off without changing the baseline ISA. A current three-model A/B
  confirmed that this decision is shape-sensitive: do not globally force the
  wider tile merely because AVX-512 is available.
- PPOCR_ENABLE_AVX512_POINTWISE_RELU8=1 opt-in enables the analogous
  eight-output `1x1 Conv+ReLU` tile for deployment-local testing. The stable
  four-output fused-ReLU path remains the default because the wider tile's
  end-to-end result is workload/cache sensitive.
- Large stride-one `3x3` feature maps can likewise use an AVX-512 eight-output
  channel tile. It shares each input vector across eight output reductions;
  the dispatcher restricts it to maps of at least 16,384 pixels and falls back
  to the four-output tile for smaller layers, odd channel tails, AVX2, NEON,
  and scalar targets. `PPOCR_DISABLE_AVX512_CONV3X3_TILE8=1` keeps the
  four-output AVX-512 path for deployment A/B. The low-level regression now
  checks both an eight-channel tile and a nine-channel ReLU tail, which also
  caught and fixed the prior un-clamped SIMD tail-channel bug.
- AVX-512 also has a dedicated unit-stride `2x2 SAME_UPPER` convolution path
  for the detector reconstruction stem. Its large valid interior uses four
  explicit FMA loads per input channel; only the final row/column runs the
  ONNX zero-padding boundary formula. It is dynamically selected only on a
  supported AVX-512 CPU, preserves the AVX2/scalar fallback, and can be A/B
  disabled with `PPOCR_DISABLE_AVX512_CONV2X2_SAME=1`.
- Equal-shape residual `1x1 Conv -> Add` pairs are fused at graph load and,
  on a runtime-eligible AVX-512, AVX2, or NEON CPU, keep the convolution
  accumulators live until the final residual add/store. This removes the
  intermediate Conv activation read/write. The executor validates the
  residual's runtime NCHW shape before selecting that kernel; scalar,
  broadcast, and unexpected-shape cases retain the existing exact
  Conv-then-Add path.
- Post-folding `Conv -> Sigmoid` and `Conv -> HardSigmoid` gate pairs are
  also fused. Their convolution result is activated in place, removing the
  generic unary output allocation; HardSigmoid uses one exact SIMD pass for
  `clamp(alpha*x+beta, 0, 1)` on AVX-512, AVX2, and NEON.
- Direct `Conv -> Add([1,C,1,1])` channel biases are folded into immutable
  convolution biases at graph load. If the Add is followed by ReLU, the three
  operations become one fused Conv+ReLU node. This covers the recognizer stem
  and avoids a full NCHW broadcast traversal and activation allocation.
- Recognition `Conv -> Sigmoid -> Mul` Swish gates are recognized after
  Conv+BatchNorm folding. The fused node writes `x * sigmoid(x)` directly into
  the Conv destination, eliminating the two branch activation tensors while
  retaining the exact FP32 operation order. The standalone BatchNorm variant
  follows the same rule when a model cannot fold the affine layer.
- `PPOCR_APPROX_GELU=1` is an opt-in PP-OCRv6 deployment mode for eligible
  model-aware fused BatchNorm+GELU planes. Its AVX-512/AVX2 dispatch is based
  on per-plane shape rather than total batch size, so batching cannot change a
  page's GELU implementation; generic GELU retains exact ONNX Erf semantics.

Model constants are no longer copied for every `Run()` invocation: they stay
in the immutable model store and execution allocates only inputs and temporary
tensors.  This substantially lowers peak transient memory usage for the
detector/recognizer.

Example cross builds (use the appropriate compiler/toolchain installed by your
environment):

```powershell
# Native ARM64 build (Visual Studio developer prompt)
cmake -S . -B build-arm64 -A ARM64
cmake --build build-arm64 --config Release

# Cross build using an ARM64 CMake toolchain file
cmake -S . -B build-aarch64 -DCMAKE_TOOLCHAIN_FILE=toolchains/aarch64.cmake
cmake --build build-aarch64 --parallel
```

The generic executor is not compiled for a host-specific ISA, avoiding
accidental illegal instructions on older x86 CPUs and on ARM targets. The
AVX-512 path remains the default on a supported CPU, while the switch is kept
for deployment-specific validation.

## ONNX Runtime comparison benchmark

The repository contains a repeatable CPU-only benchmark. It compares the
complete detector + recognition pipeline on the same PPM image, uses ONNX
Runtime's `CPUExecutionProvider`, warms up both engines, and reports mean,
median, p95, load latency and throughput.

```powershell
cmake -S . -B build -G "MinGW Makefiles"
cmake --build build --parallel
powershell -ExecutionPolicy Bypass -File .\bench\compare.ps1 -Runs 20 -Warmup 3
```

### 2026-08-14: wider correctness corpus and normalization A/B

The regression runner now accepts a UTF-8-BOM JSON case manifest as well as
plain UTF-8. This makes a small PowerShell-authored subset reproducible rather
than failing before inference. The current native validation covered all three
models against ONNX Runtime text output on English, Chinese, and `3000x80`
wide text (`3/3` cases each):
`build/accuracy_default_preprocess_ab_{tiny,small,medium}_20260814.json`.

Stress coverage also includes dense UI, tall text, and a `2560x1440`
screenshot. Tiny completed all 11 built-in cases; focused high-cost samples
completed for small (`3/3`) and medium (`2/2`), including 38 medium screenshot
regions. Reports are under `build/stress_*_current_20260814.json`.

Preprocessing retains its exact fused bilinear + normalization path. A compact
256-entry uint8 normalization lookup implementation was evaluated, but it
regressed medium on this AVX-512 host due to indexed-load/cache pressure. It
therefore remains an opt-in A/B switch (`PPOCR_ENABLE_PREPROCESS_NORMALIZE_LUT=1`)
rather than changing the production default. The paired result is recorded in
`build/bench_preprocess_lut_ab_current_20260814.json`; this keeps the default
chosen by measured end-to-end performance, not by a microbenchmark assumption.

The runtime AVX-512 dispatch remains dynamic: its current 15-run English A/B
against the AVX2 fallback measured **29.520 vs 34.133 ms** (tiny), **56.578 vs
80.106 ms** (small), and **212.366 vs 323.044 ms** (medium). See
`build/bench_avx512_dynamic_current_20260814.json`. Vulkan smoke continues to
pass on AMD Radeon 780M and admits the channel-affine+Swish and depthwise+Swish
GPU segments only when their complete transfer/dispatch/readback timing is no
slower than CPU. GPU-only correctly refuses construction until the complete
PP-OCRv6 device graph exists; it never silently runs CPU work.

`compare.ps1` now treats the requested performance target as a gate. Its
`-MinImprovementPercent` parameter defaults to `30`, and it returns a non-zero
exit code when end-to-end native latency is not at least that much lower than
the paired ONNX Runtime CPU measurement. Set it explicitly when a deployment
uses a different threshold.

`bench/ort_bench.py` requires `onnxruntime`, `numpy`, and Pillow in the Python
environment chosen by `bench/compare.ps1`. `ppocr_bench` keeps the native OCR
instance alive during all warmup/measurement iterations, so model parsing and
weight loading are excluded from steady-state latency just as they are in the
ONNX Runtime side.

### Latest measured result

The latest comparison is two consecutive `bench/compare.ps1` runs (tiny
det+rec, `build/regression_ppm/en.ppm`, `PPOCR_BACKEND=cpu`, 16 threads, 3
warmup / 20 measured, `PPOCR_APPROX_GELU` unset) captured on 2026-08-20 after
fusing the inverted-residual `1x1 expand → exact GELU → 1x1 project → residual
Add` into one spatial-tiled AVX-512 kernel (10 detector + 7 recognizer sites),
plus the prior AVX-512 stem/FPN/GEMM path. Model loading is excluded from
steady-state time. Figures are copied from those two JSON blobs; they are not
averaged into a third number.

| Run | Engine | Mean latency | Throughput | C++ / ORT | Latency improvement |
|---|---|---:|---:|---:|---:|
| 1 | Pure C++ PP-OCRv6 | 20.972 ms | 47.682 FPS | 0.208x | 79.16% |
| 1 | ONNX Runtime 1.28 CPUExecutionProvider | 100.631 ms | 9.937 FPS | | |
| 2 | Pure C++ PP-OCRv6 | 20.973 ms | 47.681 FPS | 0.209x | 79.15% |
| 2 | ONNX Runtime 1.28 CPUExecutionProvider | 100.588 ms | 9.942 FPS | | |

Both runs used `faster_engine = ppocr_cpp`. Native median/p95 were
20.884/23.443 ms and 20.376/23.720 ms. Relative to this change's pre-change
native ceiling of 30.280 ms, the pair is 30.7% / 30.7% lower latency
(each mean ≤ 21.196 ms). This is a host-specific result, not a portable
performance guarantee; rerun on the deployment CPU before capacity planning.

The previous pair after AVX-512 stride-2 tiles / fused MaxPool+Concat / FPN
Concat+Conv was 24.721 / 24.887 ms native vs 125.828 / 102.010 ms ONNX Runtime
(80.35% / 75.60%). The AVX2 exact-Erf GELU pair was 26.397 / 26.357 ms vs
99.273 / 111.544 ms (73.41% / 76.37%). The longer 60-run snapshot (2026-08-11,
8 warmup) on `build/en.ppm` was 33.074 ms native vs 138.359 ms ONNX Runtime.
Keep those as historical context only; the table above is the current
`compare.ps1` pair.

Reproduce the benchmark with:

```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File .\bench\compare.ps1 `
  -Runs 20 -Warmup 3 -Threads 16 -Backend cpu `
  -NativeExe .\build-gpu-only-final\ppocr_bench.exe
```

`bench/compare.ps1` drives the tiny pair. For small/medium, invoke
`build/ppocr_bench.exe` and `bench/ort_bench.py` directly with their matching
models and dictionary, as shown by the commands below.

### Current three-model CPU comparison

After the ARM/NEON GEMM bias-broadcast correction, the current x86 AVX-512
build was rerun on `build/en.ppm` with `PPOCR_BENCH_THREADS=16`, three warmups,
and ten timed runs per engine. The native and ONNX Runtime runs use the same
detector/recognizer pair, complete OCR pipeline, image, thread budget, and
warmup/run counts. This is a fresh shared-host snapshot rather than a portable
guarantee, but it confirms that all three supported PP-OCRv6 sizes remain well
ahead of ONNX Runtime CPUExecutionProvider:

| Model | Pure C++ mean | ORT mean | C++ / ORT | C++ latency reduction |
|---|---:|---:|---:|---:|
| tiny | 29.757 ms | 142.158 ms | 0.209x | 79.1% |
| small | 63.395 ms | 198.205 ms | 0.320x | 68.0% |
| medium | 191.198 ms | 374.134 ms | 0.511x | 48.9% |

The corresponding native throughput was 33.606, 15.774, and 5.230 FPS,
compared with ORT's 7.034, 5.045, and 2.673 FPS. Re-run the paired commands on
the deployment hardware before using these values for capacity planning.
The scalar/NEON GEMM fallback was also corrected to broadcast the bias by
output column (the same semantics as AVX-512/AVX2), rather than incorrectly
using one bias value for a whole output row. This is a correctness fix for
ARM targets and does not change the dynamically dispatched x86 measurements.

### 2026-08-13: refreshed long three-model CPU/ORT comparison

After the AVX-512 stride-one `3x3` eight-output tile was revalidated as the
default on all three model sizes, the complete `en.ppm` OCR pipeline was
rerun with 16 native/ORT CPU workers, five warmups, and 40 timed iterations
per engine.  This is the longer same-host measurement; it includes detector,
crop construction, recognizer, and decoding, but excludes model loading from
steady-state latency.  ORT used CPUExecutionProvider only.

| Model | Pure C++ mean | ORT mean | C++ / ORT | C++ latency reduction |
|---|---:|---:|---:|---:|
| tiny | 32.244 ms | 119.475 ms | 0.270x | 73.0% |
| small | 67.857 ms | 151.359 ms | 0.448x | 55.2% |
| medium | 220.768 ms | 365.267 ms | 0.604x | 39.6% |

All three exceed the repository's 30% paired CPU performance gate on this
host.  The benchmark samples remain machine-specific and ORT has wider timing
variation on this shared Windows host; full raw measurements are retained in
`build/compare_three_models_tile8_long_20260813.json`, so this table is not a
cross-platform guarantee.

### 2026-08-11: refreshed three-model exact CPU comparison

A fresh paired short run on `regression_ppm/en.ppm` used the same complete
OCR pipeline, matching 16-thread budgets, three warm-ups, and ten timed runs.
AVX-512 was dynamically selected on this x86 host; no approximation mode was
enabled.  The native and ORT measurements were executed separately on a shared
Windows machine, so these are reproducible evidence for this host閳ユ攺ot a
portable capacity promise.

| Model | Pure C++ mean | ORT mean | C++ / ORT | C++ latency reduction |
|---|---:|---:|---:|---:|
| tiny | 31.251 ms | 129.990 ms | 0.240x | 76.0% |
| small | 70.912 ms | 201.911 ms | 0.351x | 64.9% |
| medium | 199.734 ms | 409.505 ms | 0.488x | 51.2% |

This refresh keeps all three PP-OCRv6 sizes above the requested 30% native
CPU latency advantage on the same exact input. It follows the recognizer
executor lifetime fix that now executes `FusedBatchNormSwishAdd` in the
terminal-CTC path as well as the ordinary graph executor. The path reuses the
dying BatchNorm activation for affine + exact Swish + shortcut Add, reducing
the live feature-map frontier for small/medium crop batches. A profile confirms
the node as `ctc:FusedBatchNormSwishAdd:BatchNormalization.2_swish_add`.

The batch/memory follow-up used six varied PP-OCRv6 tiny inputs閳ユ攩rom `16x16`
through a `3000x80` strip閳ユ敋ith English, Chinese, and mixed content. Serial and
`RecognizeBatch()` outputs were identical (10 total results); bounded two-page
scheduling measured **134.218 ms serial / 86.501 ms batch (1.552x)**. A
medium-model memory cycle spanning `16x16`, mixed text, and a dense
`1920x1080` page used 369,672,192 private bytes after warm-up and finished at
+3,145,728 bytes. This is a Windows allocator high-water observation, not a
leak assertion; it bounds the varied-shape batch check to a small transient
increase rather than retaining every crop workspace.

## Vulkan backend status

The public API now exposes `ppocr::Backend::{cpu_only,gpu_only,hybrid}` and
`QueryBackendInfo()`. Runtime Vulkan probing is compiled from vendored Khronos
headers and dynamically loads `vulkan-1.dll`, so the binary does not require a
Vulkan SDK installation on its deployment machine. A generated SPIR-V compute
shader and smoke executable validate FP32 Add/Sub/Mul/Div building blocks over
scalar, non-workgroup-aligned, and recognizer-scale tensor sizes.
On this host, `ppocr_demo --backend-info` and `ppocr_vulkan_smoke` report the
AMD Radeon 780M compute queue and pass those correctness checks.

`gpu_only` deliberately fails if a complete PP-OCRv6 GPU graph is not
available; it never silently falls back to CPU. `hybrid` now connects verified
equal-shape in-place Add/Sub/Mul/Div executor nodes and eligible `1x1` Conv
nodes to Vulkan. For NCHW recognition tensors it maps N to the shader's Y
dimension, so a same-width crop batch uses one submission instead of one
submission per crop. A cached
per-shape/per-operation admission benchmark validates results and includes
host/device transfer, dispatch, and readback before it may select GPU; all
other operators remain on the portable CPU path. The Vulkan loader, instance,
device, queue, pipeline, descriptor set, command buffer, and grow-only
host-visible buffers are persistent, avoiding a per-dispatch Vulkan rebuild.

### 2026-08-12: depthwise GPU activation fusion

The batched Vulkan depthwise primitive now fuses its directly following
`ReLU` or exact `Swish` into the convolution writeback.  This covers the
common PP-OCRv6 MobileNet depthwise-gate shape without creating a CPU-side
activation pass or an intermediate host tensor.  The runtime still performs
shape-specific full-boundary admission (H2D, dispatch, fence, D2H, and CPU
SIMD comparison), and the new Vulkan smoke case validates batched odd-size
depthwise+Swish values against the scalar reference.  It is therefore a
batch-capable hybrid building block, not a claim that the still-partial GPU
graph is universally faster; `gpu_only` remains unavailable.

On the local Radeon 780M, the `N=4,C=32,H=31,W=29,3x3` admission sample was
**0.36344 ms GPU vs 0.62156 ms CPU**, so hybrid selected the fused GPU path.
The unfused depthwise sample in the same run was **0.33214 ms GPU vs 0.34084
ms CPU**. These figures include all transfers and synchronization and are
recorded as a per-segment comparison, not an end-to-end OCR claim.

The same depthwise kernel now also has a fused exact HardSwish variant for
PP-OCRv6 MobileNet blocks.  Its smoke admission sample measured **0.32760 ms
GPU vs 0.32076 ms CPU**, so hybrid correctly retained CPU for that shape;
the decision is dynamic rather than a blanket GPU preference.

### 2026-08-12: five-size batch and memory recheck

After the depthwise fusion work, CPU and hybrid `RecognizeBatch()` were
rechecked against serial execution on five deliberately different inputs:
`16x16`, English text, low-contrast text, a `3000x80` ultra-wide strip, and
CJK. All three official model sizes preserved exactly the same nested OCR
results. With one page worker, `det_batch_size=4`, and `rec_batch_size=4`, the
CPU measured serial/batch pairs were tiny **204.163/190.580 ms (1.071x)**,
small **392.496/375.710 ms (1.045x)**, and medium
**1260.420/1304.108 ms (0.967x)**. This mixed-shape result confirms that
batching is bounded-capacity scheduling rather than an unconditional latency
promise. The corresponding hybrid run was also result-identical (tiny
**204.311/210.308 ms**, small **436.332/411.353 ms**, medium
**1420.791/1461.834 ms**).

For compatible same-shape work, eight `en.ppm` pages with `det_batch_size=8`
and `rec_batch_size=8` measured **1.485x** CPU page-batch throughput for tiny
and **1.201x** for small. Medium did not benefit in that shared-host sample
(**0.911x**), and is therefore left with the conservative defaults rather
than promoting a batch size that regresses it. A two-cycle five-size tiny
memory soak started from **137,195,520 bytes** after warm-up and ended at
**+32,768** then **-151,552 bytes**, showing no retained per-shape activation
frontier.

Decoded-text comparison against ONNX Runtime was also rerun after these
changes for tiny, small, and medium on four diverse dimensions (`16x16`,
English, low-contrast, and `3000x80`): all **12/12** cases passed.

### 2026-08-12: GPU-aware batch regression refresh

The GPU and batch paths were rerun after the latest Vulkan/CPU optimization
pass. The public `RecognizeBatch()` API uses an exact-shape `[N,3,H,W]`
detector batch and same-width `[N,3,48,W]` recognizer crop batch; it never
rescales or pads unrelated page sizes merely to enlarge `N`. `det_batch_size`,
`rec_batch_size`, `rec_batch_width_bucket`, and `image_batch_parallelism`
remain the bounded controls, with `ppocr_batch_smoke` checking every batched
result against serial OCR before timing it.

On the local AVX-512 / Radeon 780M host, eight identical `en.ppm` tiny-model
pages (`det_batch_size=8`, `rec_batch_size=8`, one page worker) returned the
same 16 decoded results as serial execution. The measured CPU serial/batch
pair was **329.520 / 201.876 ms (1.632x)**; hybrid was **382.402 / 237.445 ms
(1.610x)**. A five-size small-model check (`16x16`, English, low contrast,
`3000x80`, CJK; batch sizes 4) likewise preserved all 9 results: CPU
**520.065 / 512.529 ms (1.015x)** and hybrid **438.718 / 427.016 ms
(1.027x)**. The mixed-size figures are intentionally not presented as a
universal throughput guarantee.

The refreshed Radeon Vulkan primitive suite passed numerical validation for
every existing batch primitive. Its strict full-boundary probes again admitted
the heavier fused work where it won (affine+Swish **0.16926 ms GPU vs 0.31296
ms CPU**; depthwise+Swish **0.36218 vs 0.66808 ms**; depthwise+HardSwish
**0.36482 vs 0.37636 ms**) and retained CPU for transfer-bound `1x1` Conv,
ordinary/transpose Conv, resize+add, and GEMM. This is intentional dynamic
selection: `gpu_only` remains explicitly unavailable until the complete
detector, recognizer, transformer, and CTC graph are device-resident.

### 2026-08-12: three-model CPU/GPU batch baseline refresh

The current pure-C++ build was rebuilt and rechecked on the complete
tiny/small/medium model set. The SIMD kernel suite passed under normal
AVX-512 dispatch and again with `PPOCR_DISABLE_AVX512=1`, exercising the AVX2
fallback. The Vulkan smoke suite passed on the Radeon 780M, including batched
binary/broadcast/affine/pointwise/depthwise/ordinary-convolution/transpose/
resize-add/GEMM primitives. In that run the full-boundary admission policy
selected Vulkan for depthwise (**0.35698 vs 0.36624 ms**), depthwise+Swish
(**0.35930 vs 0.59468 ms**), depthwise+HardSwish (**0.34864 vs 0.41902 ms**),
and affine+Swish (**0.17778 vs 0.34676 ms**); all transfer-bound primitives
stayed on CPU. The decision is shape-local and dynamic, so it lowers CPU work
without forcing a slower device boundary.

Decoded-text parity against ONNX Runtime passed **4/4** diverse inputs for
each model (tiny, small, medium): `16x16`, English, low contrast, and an
ultra-wide `3000x80` strip. A five-size serial-versus-batch gate for medium
also preserved all 9 results in CPU and hybrid modes. Its measured CPU pair
was **1257.378 / 1292.344 ms** and hybrid was **1266.158 / 1279.121 ms**;
this heterogeneous batch did not improve latency, so it is recorded as a
correctness/capacity check rather than a throughput claim.

For an `en.ppm` single-page steady-state snapshot (two warmups, five runs,
16 kernel workers), current CPU means were tiny **38.755 ms**, small
**73.192 ms**, and medium **208.209 ms**. Hybrid means were **34.575 ms**,
**75.141 ms**, and **238.014 ms** respectively. The short shared-host sample
therefore supports hybrid for tiny when the admitted GPU segments help, while
CPU remains the sensible choice for small/medium on this UMA device. These
numbers are host-specific and do not change the explicit `gpu_only` contract.

### 2026-08-12: hybrid admission re-audit

The next GPU-admission pass measured the current single-primitive boundaries
again instead of lowering thresholds blindly. On this Radeon 780M, batched
depthwise+Swish remained a useful GPU segment (**0.32756 ms GPU vs 0.57274 ms
CPU**), and the depthwise+HardSwish sample also crossed the no-slower rule
(**0.32540 vs 0.36794 ms**). Pointwise Conv, ordinary Conv, transpose Conv,
resize/add, and GEMM samples remained transfer/synchronization-bound, so they
continue on CPU. This preserves the hybrid contract: GPU is selected whenever
the complete measured boundary is equal or faster, not merely because a
shader exists.

The current source also retains conservative size guards before invoking an
admission microbenchmark. A short experiment removing them caused extra
small-shape probes and worse end-to-end tiny behavior on this UMA device, so
the guards were retained. Fresh post-audit checks passed kernel SIMD smoke,
the full Vulkan smoke suite, three-model four-size ONNX Runtime decoded-text
parity (**12/12**), five-size serial/batch output identity, and a medium
hybrid two-cycle memory soak. The latter began at **494,878,720 bytes** after
warm-up and ended **9,105,408** and **11,042,816 bytes below** baseline.

### 2026-08-12: GPU batch revalidation

GPU execution remains batch-aware: equal-width recognition crops are one NCHW
Vulkan dispatch (the shader maps crop `N` to its Y dimension), while the CPU
page scheduler bounds concurrent detector/recognizer workspaces. The pointwise
Conv shader stages compact immutable `[M,C]` weights in workgroup memory and
handles normal, ReLU, Swish, Sigmoid, and residual-Swish forms without an
intermediate activation. Hybrid admission still measures the entire H2D +
dispatch + D2H path for each shape and uses GPU only when the validated result
is no slower than the CPU SIMD counterpart.

### 2026-08-12: exact-shape detector batching

`RecognizeBatch()` now also coalesces pages whose normal PP-OCR detector
resize dimensions are exactly equal. This is a real `[N,3,H,W]` detector
batch: no document is padded, and each result is independently DB
postprocessed from its own probability-map slice. `Options::det_batch_size`
(default `4`) bounds the detector activation frontier; the smoke harness
accepts `PPOCR_DET_BATCH_SIZE` for A/B runs. CPU batch kernels therefore see
useful detector `N` work in addition to the existing same-width recognizer
crop batches.

The current hybrid path deliberately keeps this new detector batching at
`N=1`. Its Vulkan coverage is still partial and synchronous, so applying an
unproven large detector GPU segment would be transfer/admission-bound rather
than an optimization. CPU-only detector batches are available now; hybrid
continues to batch recognizer crops and only admits validated no-slower GPU
segments. This preserves the explicit `gpu_only` contract: full GPU OCR is
not claimed until detector, recognizer, transformer, and CTC stay
device-resident.

On this AVX-512 / Radeon 780M host, tiny PP-OCRv6 over eight identical
`en.ppm` pages (recognizer batch `8`, one worker) produced exactly the same
16 OCR results as serial inference. The steady-state smoke measurement was
**323.281 ms serial / 195.744 ms batch (1.652x)** with `PPOCR_DET_BATCH_SIZE=8`,
versus **330.019 / 247.684 ms (1.332x)** with detector batching forced to one.
The hybrid check preserved output identity at **311.225 / 243.242 ms
(1.279x)**; as designed, it retained singleton detector dispatches. A varied
five-page CPU run (`16x16`, English, low contrast, `3000x80`, and dense
`1920x1080`) stayed correct but measured only **1.019x**, so equal-shape
coalescing is a throughput tool for compatible page sets閳ユ攺ot a universal
mixed-document latency guarantee. Tiny, small, and medium each also retained
their four-image C++/ONNX Runtime decoded-text parity gate after the change.

### 2026-08-12: FPN resize-plus-add write fusion

The detector loader now recognizes the exported FPN sequence `Resize(nearest,
asymmetric floor, 2x) -> Add(equal-shape residual)` and replaces it with a
single `FusedNearestResizeAdd` node. Its NCHW implementation writes each
replicated source value directly into the final residual-sum allocation. This
removes the enlarged temporary feature map, its allocation, and its second
full read/write traversal. The fusion is deliberately limited to the exact
2x detector geometry; all other ONNX Resize combinations retain the generic
executor.

The local profile confirms all three recursive FPN additions are fused
(`Resize.0_add`, `Resize.1_add`, `Resize.2_add`). The post-change five-run
comparison on `en.ppm` (2 warm-ups, 16 benchmark threads) measured:

| Model | C++ hybrid mean | ORT CPU mean | C++ latency advantage |
|---|---:|---:|---:|
| tiny | 32.104 ms | 148.088 ms | 78.32% |
| small | 66.087 ms | 237.150 ms | 72.13% |
| medium | 259.850 ms | 454.815 ms | 42.87% |

This is one host's paired measurement rather than a general hardware claim.
Decoded-text parity against ONNX Runtime passed for each model on four varied
inputs (`16x16`, normal English, low contrast, and a `3000x80` strip). A
medium two-cycle varied-shape soak (`16x16`, English, low contrast, strip,
and `1920x1080` UI) ended at **-77,824 bytes** versus its warmed 391,249,920
byte private-memory baseline; this is allocator reuse evidence, not a claim
of zero memory use or a leak proof.

### 2026-08-12: runtime-SIMD FPN resize-add kernel

`FusedNearestResizeAdd` now calls a dedicated kernel rather than using scalar
replication inside the graph executor. The new kernel performs the 2x
replication and both residual-row additions in the same final stores, with
runtime AVX-512, AVX2, ARM NEON, and scalar fallback implementations. It is
covered by odd-width and batched low-level smoke cases, and
`PPOCR_DISABLE_RESIZE2X_ADD_SIMD=1` provides a narrowly scoped scalar A/B
path without disabling the other inference SIMD kernels.

On the local AVX-512 machine, a dense `1920x1080` tiny page (123 detected
crops, 2 warm-ups / 5 timed runs) measured **264.174 ms** with this SIMD
path versus **265.651 ms** through the scalar resize-add fallback (about
**0.56%** faster). This small but repeatable bandwidth-path improvement is
reported separately because whole-page timing remains sensitive to crop-count
and host load; correctness was also checked with AVX-512 disabled.

The refreshed local Radeon 780M smoke run passed all binary, broadcast,
affine, pointwise, residual, odd-plane, batched, and parameterized
`Conv+HardSigmoid` graph primitives. Its measured admission samples were:
affine+Swish **0.16466 ms GPU / 0.29544 ms CPU** (admitted), standalone `1x1`
Conv **0.14536 / 0.01952 ms**, and residual `1x1` Conv+Swish **0.18884 /
0.10648 ms** (both correctly retained on CPU). A tiny `en.ppm` pipeline check
(1 warm-up, 8 runs) measured **30.122 ms CPU** and **29.521 ms hybrid**; output
stayed identical. This small 2.0% difference is host-noise-sensitive and is
not presented as a general GPU speed claim.

### 2026-08-12: three-model, diverse-size regression

The CPU executor now has a dedicated `1x1 Conv -> HardSwish` path for the
canonical detector squeeze-excitation sequence. It writes convolution output
directly to the final activation allocation, then applies the existing
runtime-selected AVX-512/AVX2/NEON/scalar HardSwish kernel in place. This
preserves the exact `Conv -> HardSigmoid -> Mul` FP32 result while avoiding the
generic graph's temporary Conv activation. The Vulkan pointwise shader also
now covers parameterized `Conv+HardSigmoid`; its alpha/beta are passed as raw
FP32 push constants and remain behind the same complete-transfer admission
rule.

After the change, C++ versus ONNX Runtime decoded-text parity passed **7/7**
tiny samples covering English, CJK, blank, `16x16`, `3000x80`, low-contrast,
and mixed-content inputs. Small and medium also passed the ONNX Runtime text
gate on `en.ppm`. Kernel and Radeon 780M Vulkan smoke suites passed, including
the batched odd-plane pointwise and new HardSigmoid case.

On `en.ppm` (2 warm-ups, 10 timed runs, AVX-512 host), the measured
end-to-end comparison was:

| Model | C++ CPU mean | ONNX Runtime CPU mean | C++ advantage |
|---|---:|---:|---:|
| tiny | 29.159 ms | 121.267 ms | 4.16x faster |
| small | 61.785 ms | 184.410 ms | 2.98x faster |
| medium | 190.529 ms | 360.038 ms | 1.89x faster |

These measurements use the repository's ORT benchmark with the same image and
CPU execution provider; they are host-specific, but demonstrate a consistent
gain for all three PP-OCRv6 model sizes. A multi-size page-batch regression
(`16x16`, `3000x80`, and dense `1920x1080`) preserved output identity for
tiny/small/medium. Its throughput was 1.035x for tiny and 0.982x/0.975x for
small/medium on this mixed workload, so batching remains a bounded-capacity
feature rather than a universal latency claim. A medium two-cycle mixed-size
memory soak ended only **+4,345,856 bytes** above its 374,845,440-byte warmed
baseline.

### 2026-08-12: CTC LayerNorm activation reuse

`RunCtcTop1()` is the recognizer path used by public OCR calls. It now reuses
the input allocation for a fused transformer `LayerNorm` when that input has
one remaining consumer. LayerNorm first completes the mean/variance reductions
for a row and only then writes that row, so the source/destination alias is
safe and preserves the existing FP32 expression. This removes eligible
`[N,T,C]` temporary activations in the small and medium recognizers without
changing model scope or introducing a runtime dependency.

The feature has an A/B escape hatch, `PPOCR_DISABLE_CTC_LAYERNORM_INPLACE=1`,
for deployment validation. On the local AVX-512 host it produced modest
steady-state benefit on the short `en.ppm` workload (medium 225.182 ms enabled
versus 227.161 ms disabled, 2 warm-ups / 10 runs); actual benefit is expected
to be more useful for activation pressure on dense crop batches. The full
post-change regression passed kernel SIMD smoke, Radeon 780M Vulkan smoke,
the seven-image tiny ONNX Runtime text gate, and small/medium ONNX Runtime
text parity on `en.ppm`. The small/medium three-size batch gate also preserved
all 121/124 decoded results. `gpu_only` remains explicitly unavailable until
the entire detector, recognizer, transformer, and CTC pipeline is
device-resident; hybrid continues to admit each batch-shaped Vulkan segment
only when end-to-end transfer, dispatch, and readback are no slower than CPU.

### 2026-08-12: hybrid batch verification harness

`ppocr_batch_smoke` now accepts `PPOCR_BACKEND=cpu|hybrid|gpu` and the
recognizer batch controls `PPOCR_REC_BATCH_SIZE` and
`PPOCR_REC_PARALLELISM`. This makes the same serial-versus-page-batch
bit-for-bit output gate usable for the GPU-enabled hybrid path, instead of
testing only CPU scheduling. `gpu` retains the library's explicit failure
contract while the full PP-OCRv6 graph is not Vulkan-resident.

On the local AVX-512 / Radeon 780M host, tiny PP-OCRv6 with four varied pages
(`en`, `line_en`, `mixed`, `zh`), recognizer batch size 4 and one recognizer
worker produced eight identical OCR results in both serial and page-batched
modes. The measured scheduler results were:

| Backend | Serial | Page batch (parallelism 2) | Speedup |
|---|---:|---:|---:|
| CPU | 120.904 ms | 81.793 ms | 1.478x |
| Hybrid | 147.315 ms | 104.593 ms | 1.408x |

The hybrid result is intentionally reported as measured, not as a GPU win:
on this UMA GPU the available Vulkan segments are partial and transfer-bound
outside the admitted affine+Swish shape. The key correctness result is that
batch output remains identical while all runtime GPU selection stays
per-shape, batched, and no-slower-than-CPU. Reproduce with:

```powershell
$env:PPOCR_BACKEND='hybrid'
$env:PPOCR_REC_BATCH_SIZE='4'
$env:PPOCR_REC_PARALLELISM='1'
.\build\ppocr_batch_smoke.exe DET.onnx REC.onnx dict.txt 2 `
  .\build\regression_ppm\en.ppm .\build\regression_ppm\line_en.ppm `
  .\build\regression_ppm\mixed.ppm .\build\regression_ppm\zh.ppm
```

### Batch scheduling and current GPU evidence

`OCR::RecognizeBatch(const std::vector<Image>&)` is now the public page-batch
API. It preserves image/result order and exact per-image OCR output, while each
page keeps the existing same-width recognition-crop batching. Page images are
scheduled independently rather than padded into a single detector tensor: this
avoids wasted detector pixels and preserves PP-OCRv6's dynamic resize
semantics. `Options::image_batch_parallelism` bounds live page workspaces
(default `1`); when it is above one, the configured recognizer-worker budget
is divided among concurrently running pages so image batching does not multiply
the CPU thread count or activation frontier. `ppocr_batch_smoke` checks every
nested batch result against serial `Recognize()` before reporting timing.

On the local AVX-512 / Radeon 780M host, after warm-up, the tiny model over
four mixed PPM pages (`en`, `mixed`, `line_en`, `line_zh`) produced the same
seven OCR results in serial and batch modes. A short 2026-08-11 CPU snapshot
with page parallelism `2` measured **81.981 ms serial** and **53.183 ms batched**
(**1.541x** throughput). This is a scheduler/host-specific result, not a
promise for every page mix; the bounded default remains one page to minimize
peak memory. Reproduce it with:

```powershell
.\build\ppocr_batch_smoke.exe DET.onnx REC.onnx dict.txt 2 `
  .\build\regression_ppm\en.ppm .\build\regression_ppm\mixed.ppm `
  .\build\regression_ppm\line_en.ppm .\build\regression_ppm\line_zh.ppm
```

The Vulkan path already batches same-width NCHW crop tensors in one dispatch
and holds model-constant compact RHS/BatchNorm coefficients in persistent
buffers. The post-change Radeon 780M smoke run passed scalar, odd-sized,
four-stage chain, four-crop, broadcast, affine, and fused affine-Swish checks.
Its measured full-transfer admission samples were binary **0.29160 vs 0.01368
ms**, four-stage chain **0.16206 vs 0.02984 ms**, four-crop chain **0.14928 vs
0.02766 ms**, and channel-affine **0.13928 vs 0.01326 ms** (GPU vs CPU), so
hybrid correctly rejected those transfer-bound segments. The batched
affine-Swish segment measured **0.18272 ms GPU vs 0.30170 ms CPU** and was
therefore admitted for that exact shape. This is an isolated, verified GPU
segment; full `gpu_only` OCR remains unavailable until the convolution,
pooling, resize, transformer, and CTC graph is device-resident.
On the Radeon 780M, the latest 65,537-float repeated admission test measured
about **0.119 ms GPU versus 0.013 ms CPU**, so hybrid correctly declines this isolated
elementwise segment rather than increasing latency or CPU work. This is an
honest partial-graph capability contract, not a claim that GPU inference is
already complete.
The same persistent shader now supports a dependent chain of up to four
Add/Sub/Mul/Div operations in one dispatch, reusing device-resident operands
without intermediate host transfers. Its smoke test validates a 65,537-float
four-stage chain and applies the same full H2D/dispatch/D2H comparison. On this
Radeon 780M it measured about **0.124 ms GPU versus 0.030 ms CPU**, so hybrid again
correctly declines the GPU segment. This supplies a reusable fused-elementwise
building block for future PP-OCR graph segments without misrepresenting it as
complete GPU inference.
The Vulkan path also implements compact suffix-broadcast channel gates such as
`[N,C,H,W] op [N,C,1,1]`: the right operand is transferred once per batch
shape instead of being materialized to `N*C*H*W`. Runs whose four lanes stay
inside a channel span use `vec4`; only span boundaries use the scalar tail.
Its own full-transfer admission benchmark validates every output and retains
CPU SIMD unless the compact-GPU path wins. On this Radeon 780M, a four-item
8,192-float, 64-value-span `Mul+Add` gate measured **0.143 ms GPU versus
0.013 ms CPU**, so the policy correctly declined it. This is nevertheless
important GPU groundwork: it removes broadcast-expansion memory/traffic and
is connected to both generic and terminal-CTC in-place executor paths.

The batch path now also distinguishes the two valid compact RHS layouts:
dynamic `[N,C,1,1]`, and the much more common learned constant
`[1,C,1,1]`. The latter is now stored once in the persistent Vulkan RHS buffer
and addressed with a zero batch stride in the shader, rather than being
duplicated/uploaded once for every recognition crop. The Vulkan smoke test
executes two different four-crop activations against the same immutable
128-value RHS to verify both batch indexing and cache reuse. On the AMD
Radeon 780M, measured on 2026-08-11 after this change, the full-transfer
four-crop `8,192`-float `Mul+Add` shared-RHS building block was **0.11126 ms
GPU versus 0.00772 ms CPU**, so hybrid still (correctly) selects CPU SIMD.
This is a real batch-memory/traffic reduction, not a claim of end-to-end GPU
OCR acceleration while the surrounding graph remains CPU-resident.
The next batched GPU primitive is the exact inference affine
`x * scale + bias` used by fused BatchNorm. It keeps both `[1,C,1,1]`
coefficient vectors compact, caches immutable scale/bias pairs separately,
and transforms all same-width crops in one dispatch; it is wired into both
normal and terminal-CTC executor paths behind the same full-transfer admission
gate. The refreshed Radeon 780M smoke run (2026-08-11) measured the four-crop
`8,192`-float shared-coefficient affine at **0.10054 ms GPU versus 0.00656 ms
CPU**, so hybrid correctly retains the CPU SIMD affine path on this UMA device.
This establishes a useful batch/residency building block without pretending
that the present partial Vulkan graph makes end-to-end OCR faster.
The same compact four-crop kernel now also has an exact FP32
`BatchNorm affine -> Swish` mode, matching the recognizer's folded
`FusedBatchNormSwish` nodes. It evaluates `x * scale + bias` and its logistic
activation in one Vulkan dispatch, keeps immutable `[1,C,1,1]` scale/bias
vectors cached, and avoids returning to the CPU between the two operations.
It is connected to the hybrid executor through its own per-shape end-to-end
admission check. On the Radeon 780M on 2026-08-11, the four-crop
`8,192`-float shared-coefficient test measured **0.16744 ms GPU versus
0.37652 ms CPU**, so this heavier fused batch segment was admitted to GPU;
the smoke test validates every result against the exact CPU Swish expression.
This is a genuine batched hybrid acceleration, while the surrounding PP-OCR
graph remains CPU-resident and `gpu_only` remains unavailable.

### 2026-08-11: batched pointwise-Conv GPU experiment

The Vulkan executor now also contains a verified FP32 NCHW `1x1` Conv
primitive, with optional fused ReLU, Swish, and Sigmoid. It accepts a batch of equal-width
recognition crops in one dispatch, reads learned `[M,C]` weights and bias once
from persistent compact buffers, and writes to a separate lazily allocated
output buffer. The shader processes four adjacent spatial positions per
invocation when they remain in one output-channel plane; scalar tails preserve
odd widths and channel boundaries. The hybrid executor is wired only for
unit-stride, unpadded, ungrouped `Conv` / `Conv+ReLU` with immutable ONNX
weight+bias inputs. As with existing hybrid nodes, each `(N,C,M,plane,ReLU)`
shape is admitted only after an end-to-end H2D/dispatch/D2H benchmark validates
every FP32 result against the CPU SIMD kernel.

On the local AMD Radeon 780M, the four-crop `C=24, M=40, plane=257` smoke
case passed normal and ReLU numerical checks. Its complete-transfer admission
measurement was **23.715 ms GPU versus 0.026 ms CPU**, so hybrid correctly
rejected this pointwise segment on this UMA driver. The implementation is kept
as a portable batched GPU building block for devices/shapes where it wins; it
does not claim a GPU speedup here and does not change the `gpu_only` contract.

### 2026-08-12: fused GPU Sigmoid gate and refreshed checks

The pointwise Vulkan path now also fuses exact `Conv -> Sigmoid` (shader mode
10). This covers the detector's post-folding pointwise gate form without a CPU
activation pass or an extra activation buffer. Its public low-level API and
hybrid admission key distinguish ReLU, Swish, and Sigmoid; the admission probe
includes complete host upload, dispatch, fence wait, readback, and FP32
comparison with the CPU reference. A shape is selected only when that whole
round trip is no slower than CPU, and `gpu_only` remains unavailable until the
complete detector/recognizer/transformer/CTC graph is device-resident.

On the local AVX-512 / AMD Radeon 780M host, the refreshed Vulkan smoke test
passed the batched, odd-plane (`N=4, C=24, M=40, plane=257`) Conv+Sigmoid
numerical check together with existing binary, affine, Swish, residual, and
shared-memory-tiled pointwise checks. The same run reported:

| Segment admission sample | GPU | CPU | Hybrid selection |
|---|---:|---:|---|
| Batch affine+Swish | 0.15924 ms | 0.29874 ms | GPU |
| Batch `1x1` Conv | 0.16150 ms | 0.02062 ms | CPU |
| Batch `1x1` Conv+Swish | 0.18282 ms | 0.10534 ms | CPU |
| Batch residual `1x1` Conv+Swish | 0.14726 ms | 0.09338 ms | CPU |

These are measured partial-segment results on a UMA GPU, not an end-to-end GPU
claim. A post-change whole-pipeline smoke benchmark on `regression_ppm/en.ppm`
with the tiny model (3 warm-ups, 12 runs) measured CPU **28.584 ms** and
hybrid **27.684 ms** mean. A medium-model check (2 warm-ups, 8 runs) measured
CPU **201.388 ms** and hybrid **206.885 ms**; shared-host variation and partial
coverage mean CPU remains the sensible default for this device. Both modes
preserved OCR output correctness. The low-level AVX-512 kernel regression also
passed normally and with `PPOCR_DISABLE_AVX512=1`, verifying the AVX2 fallback.

Reproduce the verification:

```powershell
cmake --build build --parallel
.\build\ppocr_vulkan_smoke.exe
.\build\ppocr_kernel_smoke.exe
$env:PPOCR_DISABLE_AVX512='1'; .\build\ppocr_kernel_smoke.exe
Remove-Item Env:PPOCR_DISABLE_AVX512
```

### 2026-08-12: diverse-size batch and memory regression

The post-tile regression was widened beyond short English lines. With
recognizer batch size 4 and one recognition worker, the medium model preserved
serial-equivalent OCR across a `16x16` tiny glyph, a `3000x80` wide strip, and
a dense `1920x1080` UI page (124 total results). It measured **5652.199 ms
serial / 5552.878 ms batch** (`1.018x`) and a subsequent two-cycle mixed-size
memory soak used **391,716,864 bytes** after warm-up; cycles ended at
**+6,422,528** and **+184,320 bytes**. This is bounded Windows allocator
retention after the dense page, not a per-shape cache. Small-model CPU and
hybrid checks likewise preserved output parity across `16x16`, `3000x80`,
mixed-language, and dense-page cases. The current GPU graph is still partial,
so page batching is a capacity feature rather than a universal latency win;
the bounded default remains one concurrent page.

The accompanying 1-warmup/7-run `en.ppm` whole-pipeline A/B (same host,
default batch settings) was:

| Model | CPU mean | Hybrid mean | Result |
|---|---:|---:|---|
| tiny | 27.867 ms | 34.334 ms | CPU faster |
| small | 67.981 ms | 79.373 ms | CPU faster |
| medium | 205.998 ms | 214.396 ms | CPU faster |

This outcome is expected for a partial graph with synchronous round-trips;
use `PPOCR_BACKEND=cpu` on this device for current end-to-end throughput. The
six-image mixed-size page-batch regression still produced byte-identical OCR
results versus serial execution and measured **410.192 ms serial**, **373.145
ms batched** at page parallelism 2 (**1.099x**).
For a fuller pipeline A/B on the same shared Windows host (small model,
`ui_dense_1920x1080.ppm`, 119 boxes, 16 CPU workers, recognizer batch 4 /
parallelism 14, one warm-up and three timed runs), CPU measured **1044.384 ms
mean** and hybrid **1081.426 ms mean**. This short noisy sample is recorded as
an end-to-end non-regression check rather than a GPU speedup: the per-shape
admission rule continued to reject GPU segments, and hybrid's small overhead
remains a reason to use `PPOCR_BACKEND=cpu` when the deployment only needs the
current CPU graph.

After the fused Swish GPU path was added, a fresh small-model dense-page A/B
(`ui_dense_1920x1080.ppm`, 119 boxes, batch 4, parallelism 14, 1 warm-up / 3
runs) measured **1170.717 ms hybrid** and **1155.998 ms CPU**. The short run is
intentionally reported as a non-regression observation rather than an
end-to-end GPU speed claim: only eligible BatchNorm-Swish shapes are offloaded,
and scheduling/shape mix can still outweigh an individual accelerated segment.
The per-shape gate continues to preserve correctness and rejects any shape
whose complete transfer/dispatch/readback time loses to CPU.

The CPU fallback for the exact `Sigmoid`, `Swish`, and
`BatchNorm -> Swish` paths is also unrolled into four independent scalar-libm
exponentials. This improves instruction-level parallelism on x86 (including
AVX-512 hosts, where there is still no exact vector `exp` instruction) and
ARM/NEON without adopting an approximate exponential or changing the ONNX
logistic expression. The post-change tiny/small/medium ONNX Runtime text
checks remained **5/5**, **3/3**, and **3/3** respectively.
Fused `BatchNorm -> Swish` now also reuses the source activation when it is at
its final graph use. This removes one recognizer-sized output allocation and
copy on both x86 and ARM/NEON while preserving the affine, exact logistic, and
multiply operation sequence per scalar. It can be A/B tested with
`PPOCR_DISABLE_BN_SWISH_INPLACE=1`. On the shared host, a small-model
three-crop `mixed.ppm` sample was noisy and did not show a reliable latency
win (100.535 ms in-place vs 98.598 ms legacy over 15 runs), whereas medium
measured 278.592 vs 282.793 ms over 10 runs; therefore this is recorded as an
activation-memory/frontier optimization rather than a portable speed claim.

A later experiment to fuse transformer `GEMM -> exact Swish` at row granularity
was A/B tested and intentionally not retained: its benefit was not stable on
small/medium at the end-to-end level. The portable GEMM and exact Swish passes
therefore remain separate, avoiding an unsupported speed claim. This is the
same evidence-first policy used for every hybrid GPU admission.

The same shader now accepts an equal-sized batch in its Vulkan dispatch Y
dimension: independent recognizer-like crops share one command-buffer reset,
submission, completion wait, and H2D/D2H pair instead of paying those costs once per
crop. Each compute invocation handles four contiguous FP32 values (`vec4`) and
keeps only a 1--3 value scalar tail, reducing work-item scheduling overhead on
wide OCR planes. The shader updates the mapped left-hand activation buffer in
place, eliminating the former dedicated output buffer and its activation-sized
copy: this Vulkan primitive now retains two persistent buffers
(activation/result plus compact RHS), rather than three. Persistent mapped
buffers now prefer coherent host-cached memory when the Vulkan driver offers
it (with a portable coherent-memory fallback), grow with 1.5x headroom, and
resize the activation/result independently of the compact RHS. This avoids
needless buffer recreation when recognition width buckets change. Completion uses a submission fence rather
than queue-wide idle, so an individual synchronous mapped readback does not
also wait for unrelated queue work. The batch admission check validates every
item and includes complete batched transfer/dispatch/readback time; hybrid
selects it only when that time is no slower than CPU SIMD. The current Radeon
780M four-crop, 4,099-float, four-stage check measured **0.146 ms GPU versus
0.037 ms CPU** after this batch-path update, so hybrid correctly retained CPU.
This is an implemented
batched Vulkan executor path and building block, not a full GPU OCR claim until
convolution, pooling, resize, normalization, MatMul, and CTC are device-resident.

For repeatable backend A/B measurements, `ppocr_bench` accepts
`PPOCR_BACKEND=cpu|hybrid|gpu` (default: `hybrid`) in addition to
`PPOCR_REC_BATCH_SIZE`, `PPOCR_REC_WIDTH_BUCKET`, and
`PPOCR_REC_PARALLELISM`. Its JSON output records the selected backend. `gpu`
continues to fail explicitly until the full PP-OCRv6 graph is Vulkan-resident;
it never labels a CPU fallback as GPU inference.

The post-optimization Vulkan smoke test on 2026-08-11 passed scalar,
non-workgroup-aligned, 65,537-element, dependent-chain, four-item batch, and
compact NCHW suffix-broadcast batch cases on AMD Radeon 780M Graphics. The
suite now also covers an immutable shared-RHS broadcast batch over two distinct
activations. The broader native regression corpus also
passed **11/11** cases: blank, tiny, wide/tall strips, low-contrast/noisy,
rotated, dense blobs, and 1,920鑴?,080 / 2,560鑴?,440 screenshots. Text parity
against ONNX Runtime passed **5/5** mixed English/Chinese samples. Exact pixel
geometry is intentionally not a strict invariant because equivalent DB contour
rounding can differ by one pixel; recognized text and normal-tolerance
confidence parity are checked.

The CPU convolution path additionally fuses ReLU into the AVX-512/AVX2
2鑴?-valid and 3鑴? convolution stores (both stride-1 and stride-2), which
removes the subsequent full activation read/write pass for detector stems,
the reconstruction head, and the medium detector context head.
`PPOCR_DISABLE_FUSED_CONV_RELU=1` is
available solely for deployment A/B measurement; it retains the former
separate SIMD ReLU pass. On the two-line `en.ppm` workload, a 20-run tiny A/B
was **39.385 ms fused vs 40.599 ms unfused** (about 3.0% lower mean latency).
After the stride-1 extension, a 15/12/12-run A/B measured tiny at **46.692 vs
48.548 ms** (3.8% lower), small at **128.286 vs 136.103 ms** (5.7% lower), and
medium at **312.317 vs 292.898 ms**. The medium result is adverse on this
shared host, so the `PPOCR_DISABLE_FUSED_CONV_RELU` escape hatch remains for
deployment tuning; it is reported rather than presented as a portable gain.
Correctness was rechecked with AVX-512 disabled (AVX2 dispatch), five ORT
text-parity samples, and the 11-case stress corpus.

Adding 2鑴?-valid coverage was remeasured on the same `en.ppm` workload:
15-run tiny was **35.783 vs 37.773 ms** (5.3% lower), medium was **271.887 vs
272.238 ms** (0.1% lower), while small contained a single 738.612-ms scheduling
outlier (median **91.147 vs 94.687 ms**). The median is the useful comparison
for that shared-host run; all raw mean/median/p95 numbers remain available from
`ppocr_bench` JSON for deployment-specific decisions.

The executor now also transfers a dead activation's backing storage through
internal `Reshape`, `Squeeze`, and `Unsqueeze` nodes instead of copying floats
solely to change tensor metadata. It retains the existing last-use and graph
output guards, so aliases never escape into a live consumer or public result.
The terminal-CTC recognizer executor uses the same transfer path, covering the
small/medium transformer reshape boundaries rather than only the generic
executor. ORT text parity passed tiny **5/5** and small/medium **3/3** sampled
English, Chinese, and mixed pages after this change. The expanded 11-image
stress suite also passed. A size-changing memory soak across `en.ppm` and
`ui_dense_1920x1080.ppm` remained bounded: tiny stabilized around 80 MiB
private bytes (at most +3.61 MiB above its post-warmup sample) and medium
stayed below its 308.93 MiB post-warmup sample on the measured cycles; small
remained around 105--112 MiB (+6.34 MiB at most after warmup). Dense-page
native throughput was additionally exercised with 119 small-model crops
(1.422 s mean) and 122 medium-model crops (6.009 s mean), confirming the
optimized recognizer path under varied crop widths and bounded batches.

`PPOCR_PROFILE=1` now additionally emits a separate `ppocr CTC operator
profile` for the recognizer's terminal-CTC executor. This exposes the actual
small/medium transformer costs (rather than only detector `Run()` nodes) while
keeping the release path free of timing/map allocations. On the 123-crop dense
page, tiny-model batch-size tuning measured `rec_batch_size=4` at **323.479
ms**, versus 348.242 ms (8) and 339.535 ms (16); the default remains 4. A
parallelism sweep for that same batch size measured 498.854 ms (4 workers),
392.288 ms (8), 339.808 ms (14), and 329.693 ms (24). The 24-worker result is
only a small host-specific gain with a higher concurrent activation frontier,
so the library retains its more memory-efficient default of 14 workers.

The current transformer kernels add two further low-memory CPU optimizations.
Wide AVX-512/AVX2 GEMM projections now retain four output vectors in registers
across the entire K reduction, eliminating the former output load/store for
every K item; this directly targets the `FusedMatMulBias` vocabulary head.
The repeated LayerNorm `Pow(x,2)` nodes now use a SIMD square kernel and reuse
their dead centered activation in place, avoiding both `powf` and a second
`[N,T,C]` allocation. The latter is exact multiplication, not an approximate
math mode. On the latest 16-worker 20-run paired `en.ppm` measurement, the
current release measured **41.718 ms tiny, 93.439 ms small, and 270.362 ms
medium**, against ONNX Runtime CPUExecutionProvider **144.039, 272.186, and
640.506 ms** respectively. That is **0.290x / 0.343x / 0.422x** C++/ORT
latency (about **71.0% / 65.7% / 57.8% lower** C++ latency) on this shared
host. Exact ORT text parity after these changes passed tiny **5/5** and
small/medium **3/3** English, Chinese, and mixed-image cases.

A post-change changing-size memory soak (`en.ppm` plus the 1,920鑴?,080 dense
page) remained bounded. Tiny's warm baseline was **78,929,920 bytes** and
its largest measured delta was **+1,032,192 bytes**; small's baseline was
**107,143,168 bytes** and its largest delta **+6,766,592 bytes**; medium was
at or below its **284,958,720-byte** warm baseline in both measured cycles.

The five transformer LayerNorm chains in each small/medium recognizer are now
recognized at model load and collapsed from `ReduceMean 閳?Sub 閳?Pow 閳?ReduceMean 閳?Add 閳?Sqrt 閳?Div 閳?Mul 閳?Add` into one affine last-axis kernel.
It creates only the final `[N,T,C]` activation, eliminates centered/squared/
normalized temporary tensors, and keeps the standard `sqrt`, division, gamma,
and beta operation ordering. On the same 16-worker 20-run `en.ppm` protocol,
the resulting C++ measurements were **36.683 ms tiny, 75.827 ms small, and
229.388 ms medium**, while the paired ORT figures remain **144.039, 272.186,
and 640.506 ms**. That is **74.5% / 72.1% / 64.2% lower latency** for the
native path on this host. Post-change text parity passed tiny **5/5** and
small/medium **3/3**. A fresh multi-size soak measured small from a
**109,473,792-byte** warm baseline with a maximum **+4,493,312-byte** delta;
medium began at **276,164,608 bytes** with a maximum **+929,792-byte** delta.

`MatMul + bias + Sigmoid + Mul` gates in the small and medium transformer MLPs
are also fused into a single bias-initialized MatMul followed by in-place Swish;
each model matches this graph pattern twice. It removes the intermediate bias
and sigmoid activations without changing FP32 operation order. Text parity
after that change passed tiny **5/5** and small/medium **3/3** against ONNX
Runtime. A fresh CPU-only 16-worker sanity run (5 warmups / 20 runs, subject
to documented Windows scheduling variation) measured **43.778 ms tiny,
97.592 ms small, and 277.404 ms medium** on `en.ppm`; the established paired
ORT baseline above remains the authoritative cross-engine comparison.

The fused MLP path is now additionally specialized for its actual PP-OCRv6
rank-2 constant projection: it derives the flattened row count directly and
avoids generic batch-shape/stride bookkeeping before invoking GEMM, then
applies Swish in the same destination. Generic broadcasted `MatMul` retains
the portable implementation, but its batch strides are now computed once per
node instead of allocating them once per batch item. The latest validation
covered all three models across **16 PPM inputs**: tiny and small completed all
16 in one run, while medium completed all 16 in two timeout-bounded eight-image
runs. The corpus spans 16x16, 80x3000, 3000x80, noisy/low-contrast, rotated,
dense, blank, Chinese/English/mixed, and 1920x1080 / 2560x1440 screenshot
inputs. Exact ORT text parity was rechecked as tiny **5/5** and medium **3/3**
after the specialisation; small's existing **3/3** exact gate remains passing.

The AVX-512 `1x1 Conv + residual Add` path now has a real eight-output
register tile rather than issuing two four-output kernels. Each 16-float input
vector is loaded once and contributes to eight independent output channels;
the residual is added only after each channel reduction, retaining the ONNX
`Conv` then `Add` operation order. On the same 16-worker, 5-warmup/20-run
medium `en.ppm` A/B, the enabled tile measured **296.941 ms** (median 295.073)
versus **317.217 ms** (median 317.510) with
`PPOCR_DISABLE_AVX512_POINTWISE8=1`: a **6.4% mean** reduction on this host.
The current post-change CPU sanity sweep measured **42.518 ms tiny, 86.810 ms
small, and 296.941 ms medium**; shared-host variance means the established
paired C++/ORT table remains the authoritative engine comparison. Tiny 5/5
and medium 3/3 ORT text parity were repeated after the kernel change; the
medium three-size memory soak (16x16, 3000x80, dense 1920x1080) started at
**330,235,904 bytes** and ended one cycle at **+4,177,920 bytes**. Vulkan's
scalar, odd-size, chain, batch, and compact-broadcast smoke cases also passed.

Recognizer attention transpose now has exact rank-4 specialisations for the
two PP-OCRv6 permutations `[0,2,1,3]` and `[0,1,3,2]`. The former moves its
contiguous final feature vector with `memcpy`; both avoid the generic
per-float four-axis divide/modulo index decoder. After the change, ORT text
parity passed tiny **5/5**, small **3/3**, and medium **3/3**. On the current
16-worker 5-warmup/20-run CPU samples, tiny/small/medium measured **42.771 /
87.300 / 261.648 ms** on `en.ppm`. The current Vulkan smoke run also passed;
the Radeon 780M admission samples remained GPU-slower end to end and hybrid
therefore retained CPU SIMD rather than claiming an unsupported speedup.

Attention Q/K/V splitting now also has a safe contiguous `Slice` fast path:
when ONNX Slice selects an axis-0 unit-stride range while preserving every
remaining axis, it transfers the complete `[axis-0 row]` span with one
`memcpy` rather than doing per-value stride division/modulo. This exactly
matches the stacked Q/K/V slices used by the small and medium recognizers;
all non-contiguous or non-unit-step layouts retain the general ONNX Slice
implementation. The post-change exact ORT gates passed tiny **5/5**, small
**3/3**, and medium **3/3**. A wider eight-case small-model corpus was also
run; six cases passed (blank, English, Chinese, mixed, line, and noisy), while
the deliberately difficult dense-screen and rotated-text samples are currently
recorded as known OCR-output divergences from ORT rather than excluded or
misreported as a passing parity gate.

The recognizer's remaining final-head layout exchange `[N,T,C] -> [N,C,T]`
now also uses a direct three-axis loop instead of generic shape/stride index
decoding. Together with the rank-4 attention paths, this covers every fixed
transpose permutation used by the shipped tiny/small/medium recognizers while
retaining the generic implementation for other ONNX graphs. The post-change
parity gates again passed tiny **5/5**, small **3/3**, and medium **3/3**.
The medium short benchmark remained shared-host variable (a fresh 20-run
sample measured 302.714 ms), so no additional end-to-end speed claim is made
from that isolated run; the change is retained for lower overhead on the
repeated transpose-heavy recognizer paths.

The latest backend A/B sample on the two-line tiny workload (`en.ppm`, 3
warmups / 15 runs) measured **41.533 ms hybrid** versus **42.553 ms CPU**.
This small 2.4% difference is within normal shared-host variability: the
Radeon admission measurements above declined all tested segments, so it must
not be interpreted as a standalone GPU OCR speedup. On the 123-crop dense
page (2 warmups / 7 runs), hybrid measured **311.501 ms** and CPU **314.929
ms**; both paths preserve exact CPU inference for the parts of PP-OCRv6 not
yet resident on Vulkan.

After the LayerNorm/transformer update, the denser 123-crop 1,920鑴?,080 workload
was repeated for 3 warmups / 10 runs: **299.313 ms hybrid** (median 297.708)
and **293.864 ms CPU** (median 292.125). The Radeon 780M's independently
measured admissions still rejected the individual Vulkan elementwise segments
(for example the compact broadcast batch was 0.106 ms GPU versus 0.010 ms
CPU), so this close whole-pipeline comparison is recorded as shared-host
variance rather than claimed GPU acceleration.

After the Vulkan batch-buffer/fence update, the same 123-crop dense page was
rechecked for 3 warmups / 10 runs: **346.120 ms hybrid** (median 342.037) and
**347.319 ms CPU** (median 345.431). This near tie is shared-host variance,
not GPU acceleration: all current per-shape admissions still rejected the
Radeon 780M elementwise segments. The smoke suite passed scalar, odd-length,
fused-chain, compact broadcast, and four-crop batch cases; its current
end-to-end admission samples were 65,537-float binary **0.256 vs 0.028 ms**,
four-stage chain **0.169 vs 0.032 ms**, four-crop chain **0.146 vs 0.037 ms**,
and compact broadcast **0.150 vs 0.016 ms** (GPU vs CPU). `gpu_only` was also
rechecked and continues to fail explicitly because the full Vulkan OCR graph
has not yet been implemented.

After the in-place mapped-buffer update, Vulkan smoke was rerun on 2026-08-11
and passed scalar, odd-length, fused-chain, four-crop batch, and compact
broadcast batch outputs on AMD Radeon 780M Graphics. Current full-transfer
admission samples remained transfer-bound: binary **0.333 vs 0.034 ms**,
four-stage chain **0.250 vs 0.047 ms**, four-crop chain **0.133 vs 0.031 ms**,
and compact broadcast **0.161 vs 0.015 ms** (GPU vs CPU). Accordingly hybrid
still executes those isolated segments with CPU SIMD. The saved mapped-buffer
memory is real, but it is not represented as a whole-OCR GPU speedup.

The fused pointwise-convolution channel-bias path now uses the existing
runtime-dispatched scalar-broadcast kernel per NCHW plane. This removes its
scalar inner loop while preserving the exact `Conv` then `Add` order, with
AVX-512, AVX2, NEON, and portable scalar coverage selected dynamically. The
follow-up exact ORT gates passed tiny **5/5**, small **3/3**, and medium
**3/3**. A post-change 16-worker 5-warmup/20-run CPU sanity sample measured
**41.587 / 84.744 / 259.606 ms** for tiny/small/medium on `en.ppm`; Windows
scheduling varies enough that the paired C++/ORT table above remains the
authoritative cross-engine claim. A small dynamic-size memory soak across
16鑴?6, 3000鑴?0, and dense 1920鑴?080 inputs started at **110,268,416 bytes**;
two cycles ended at **-2,564,096** and **+2,109,440 bytes** from that baseline,
respectively, with no monotonic retained-shape growth.

The latest exact-mode CTC confidence scan now issues four independent
scalar-libm exponentials before adding them in the original lane order. This
preserves the 6,906-class Softmax confidence calculation and CTC output while
allowing x86 (including AVX-512-capable systems) and ARM cores to overlap exp
latency. Build plus ONNX Runtime text gates passed tiny **5/5**, small **3/3**
and medium **3/3** after the change. A fresh 16-worker, 5-warmup/20-run CPU
sanity pass on `en.ppm` measured **36.458 / 70.745 / 218.498 ms** for
tiny/small/medium. The same revision's diverse-size CPU sweep recorded:

| Model | `en.ppm` (2 boxes) | `wide_strip_3000x80.ppm` | `ui_dense_1920x1080.ppm` |
|---|---:|---:|---:|
| tiny | 33.555 ms | 23.269 ms | 278.795 ms (123 boxes) |
| small | 74.727 ms | 57.801 ms | 1136.849 ms (119 boxes) |
| medium | 221.817 ms | 150.859 ms | 4626.259 ms (122 boxes) |

Those short varied-size samples are intentionally recorded as local CPU
capacity data, not substituted for the repeated paired C++/ORT table. The
post-change small mixed-size two-cycle memory soak (16x16, 3000x80, and
1920x1080 dense UI) measured **-12,288 bytes** then **+5,222,400 bytes** from
a 109,170,688-byte baseline, showing no monotonically growing allocation per
input shape. Vulkan smoke also passed on AMD Radeon 780M Graphics; full
transfer admission remained CPU-faster (binary **0.13716 vs 0.01282 ms**,
four-stage batch **0.10178 vs 0.02728 ms**, GPU vs CPU), so hybrid correctly
continued to avoid increasing latency or CPU work with an unprofitable GPU
offload.

For AVX-512 x86 systems, the detector's hot stride-2 3x3 convolution now
forms its sixteen every-other-pixel samples from two contiguous row loads and
an AVX-512 lane permute, rather than issuing a gather for each 3x3 tap. The
packed-load implementation remains dynamically switchable with
`PPOCR_DISABLE_AVX512_STRIDE2_PACKED=1`, preserving the prior gather kernel
for any deployment where its microarchitecture wins. On this host's medium
`en.ppm` 16-worker 5-warmup/20-run A/B, packed loads measured **219.768 ms**
(median 216.588) versus **274.781 ms** (median 229.537) for gathers. The mean
is scheduler-sensitive, but both median and mean favor the packed path. Its
full exact ORT text gates passed tiny **5/5**, small **3/3**, medium **3/3**;
a medium mixed-size memory cycle (16x16, 3000x80, dense 1920x1080) ended
**98,304 bytes below** a 277,999,616-byte baseline. The Vulkan smoke suite
also remained passing; its current full-transfer admissions still correctly
select CPU for this Radeon 780M, so this CPU optimization never changes the
GPU-only or hybrid correctness contract.

An AVX-512 eight-output-channel 3x3 stride-one detector kernel shares every
interior input vector across eight output reductions while retaining scalar
border handling and the exact per-channel reduction order. A refreshed
24-run, 2-warmup CPU A/B on the current host shows the current restricted
default predicate is beneficial for all three `en.ppm` models: tiny
**32.010 -> 31.166 ms** (1.027x), small **66.480 -> 65.099 ms** (1.021x),
and medium **207.772 -> 203.876 ms** (1.019x), four-output -> tile-8.
`PPOCR_DISABLE_AVX512_CONV3X3_TILE8=1` remains the deployment fallback for
machines whose AVX-512 frequency characteristics differ. The stricter shape
gate still routes smaller layers, odd channel tails, AVX2, NEON, and scalar
targets to their existing proven paths.

The adjacent AVX-512 eight-output `1x1` projection tile was also refreshed as
a negative-control A/B, rather than broadening its predicate speculatively.
With the current `192x192` gate, four-output -> eight-output measured tiny
**32.097 -> 34.343 ms** (regression), small **70.746 -> 66.848 ms** (win),
and medium **216.021 -> 214.452 ms** (small win), using the same 2-warmup,
24-run CPU procedure. Consequently the shape-local `192x192` default and
`PPOCR_DISABLE_AVX512_POINTWISE8=1` fallback remain unchanged: a single global
threshold cannot improve every tier on this client CPU. This is deliberately
recorded rather than claiming a universal pointwise gain.

The detector FPN's common nearest-neighbour 2x resize now writes four source
pixels as eight direct replicated stores before vertically copying the row.
This keeps the exact NCHW output layout while removing per-pixel
`std::fill_n` setup in every 2x FPN plane; uncommon integer resize factors
retain the existing general replication path. A fresh default medium
16-worker 5-warmup/20-run `en.ppm` sample measured **209.972 ms** (median
210.745 ms). Exact ORT text regression again passed tiny **5/5**, small
**3/3**, medium **3/3**, and Vulkan smoke stayed passing. A two-cycle tiny
mixed-size soak across 16x16, 3000x80 and 2560x1440 inputs stayed bounded at
**+1,150,976** then **+2,654,208 bytes** relative to its 82,276,352-byte
post-warmup baseline; this represents Windows heap retention, not a
per-inference activation leak.

Recognition batch construction no longer allocates and copies a temporary
`vector<CropBounds>` for every batch. Same-width buckets now retain contiguous
crop bounds and result indices once, and `RecInputBatch` consumes a direct
span. This removes hundreds of small allocations/copies on dense pages while
preserving crop order, width bucketing, bounded batch size, and concurrent
recognizer behavior. The medium 122-box dense 1920x1080 smoke workload
completed correctly after this change (a short 2-warmup/5-run local sample
was **5541.268 ms**, subject to the documented scheduler/cache variability).
Exact ORT text parity remained tiny **5/5**, small **3/3**, medium **3/3**;
Vulkan scalar/chain/batch/broadcast smoke passed. A medium mixed-size
16x16/3000x80/1920x1080 memory cycle completed without a failure; its
post-warmup private-byte result is retained as capacity data rather than
claimed as a leak-free steady-state proof because the Windows allocator kept
10,149,888 bytes after this deliberately large dense workload.

The Vulkan binary runtime now recognizes immutable ONNX initializer operands.
When hybrid dispatches an eligible elementwise node, its RHS remains in the
persistent mapped Vulkan buffer across later activations of the same model,
avoiding a repeated host copy while retaining a separate activation/result
buffer. Dynamic RHS tensors always copy, and the existing full end-to-end
admission policy is unchanged: a cached constant does not qualify GPU unless
the whole H2D/dispatch/D2H segment is no slower than CPU. The smoke suite now
checks two distinct activations against a cached 65,537-float immutable RHS;
it passed on AMD Radeon 780M Graphics along with scalar, chain, batch and
broadcast cases. Current admissions remain CPU-faster on this UMA host, so
hybrid correctly preserves CPU SIMD rather than claiming a GPU OCR speedup.
The default CPU executor's tiny **5/5**, small **3/3**, and medium **3/3**
ORT text gates were rerun after the backend API change.

### Batched Vulkan residual pointwise block

The Vulkan backend now has a verified batched `1x1 Conv + residual Add`
primitive, with a `Conv + Add + ReLU` variant. It maps equal-width recognition
crops to Vulkan's Y dimension, keeps learned `[M,C]` weights and bias cached
when they are ONNX initializers, and uploads the residual into a separate
read-only buffer so the shader never aliases an input with its output. This
is also wired into the existing fused CPU graph nodes in `hybrid` mode, while
the shape-specific admission test includes both live activation uploads,
parameter transfer, dispatch, readback, and an FP32 CPU comparison.

On 2026-08-11, the AMD Radeon 780M smoke test validated normal and ReLU
residual pointwise results for four NCHW batches, including a 257-element
non-vector-aligned spatial plane. Its full-transfer admission measurement for
the `N=4, C=24, M=40, plane=257` `Conv+Add+ReLU` case was **19.1844 ms GPU**
versus **0.0326 ms CPU**, so hybrid correctly rejected the GPU segment. The
normal pointwise case was likewise **19.1564 ms GPU** versus **0.03072 ms
CPU**. These figures are deliberately recorded as a negative end-to-end
result: the new GPU kernel is numerically covered and batch-capable, but
standalone host-visible uploads/readback make it inappropriate on this UMA
device. `gpu_only` remains explicitly unavailable until the complete OCR
graph can execute device-resident without CPU fallbacks.

After this change, the tiny five-image C++/ONNX Runtime text regression again
passed **5/5**. The six-image mixed-size page-batch smoke (151 detected text
regions, page parallelism two) completed in **381.767 ms** versus **408.840
ms** serial, a **1.071x** scheduler speedup on the local shared host.

The residual-ReLU AVX-512 eight-output entry point was also exercised on the
same host. Its real eight-live-accumulator version passed the five-image text
gate, but did not demonstrate a stable end-to-end win: `mixed.ppm` small was
**86.242 ms** versus **82.700 ms** with the default four-output tile, and the
medium results varied between runs. It therefore continues to delegate to the
proven four-output residual-ReLU tile instead of becoming an unvalidated
default. This keeps AVX-512 dynamic and safe while avoiding a register-pressure
regression on both tiny and small workloads.

The current medium mixed-size memory soak (`16x16`, `3000x80`, `80x3000`, and
the 151-region `1920x1080` UI page; two cycles) established a post-warm-up
private-byte baseline of **347,013,120 bytes**. The next cycles were
**+303,104 bytes** and **+3,276,800 bytes**. This bounded allocator-capacity
movement is recorded as a Windows high-water observation rather than a general
leak proof; importantly it did not grow once per dynamic shape or per crop
batch.

### 2026-08-11: GPU residual-vector and non-ReLU batch fusion follow-up

The residual pointwise shader now vectorizes its common four-contiguous-spatial
lane case as well as the non-residual pointwise shader.  It performs
`1x1 Conv + residual Add (+ ReLU)` in registers and writes one final NCHW
result, retaining a scalar tail only at plane boundaries and odd widths.  The
graph loader also recognizes the strict dynamic `1x1 Conv -> equal-shape Add`
form without a following ReLU as `FusedPointwiseConvAdd`.  This matters for
the medium recognizer: a CPU profile of `mixed.ppm` observed fused non-ReLU
shortcut nodes such as `Conv.9_add`, `Conv.20_add`, `Conv.50_add`, and
`Conv.31_add`, so they now avoid a materialized Conv result plus a separate
Add pass on both CPU and eligible hybrid paths.  Initializer/broadcast Adds
remain excluded, preserving ONNX bias semantics.

The rebuilt Vulkan smoke suite passed all binary, broadcast, affine, normal
pointwise, residual pointwise, and residual-ReLU numerical checks on AMD
Radeon 780M Graphics.  Its latest complete-transfer measurements were
**21.0106 ms GPU vs 0.02642 ms CPU** for `N=4,C=24,M=40,plane=257` pointwise
Conv and **14.2844 ms GPU vs 0.03048 ms CPU** for residual+ReLU; hybrid
therefore rejected both exactly as designed.  The separate tiny five-image
C++/ORT text gate passed **5/5**, and the six-page mixed-size batch smoke
produced identical nested results in **81.021 ms** versus **122.164 ms**
serial (**1.508x**).  A short `mixed.ppm` end-to-end check measured **34.379
ms CPU** versus **40.314 ms hybrid** (one warm-up, three runs), so current
deployment guidance remains CPU for this UMA GPU; these GPU additions are
correct, batched building blocks rather than an unsupported full-GPU claim.

### 2026-08-11: Vulkan mapped-memory policy revalidation

The persistent Vulkan buffers now explicitly score host-coherent memory types
for this synchronous hybrid runtime: `HOST_CACHED` coherent memory is selected
first, then host-visible device-local coherent memory, then a portable coherent
fallback. This is not merely a theoretical ordering. An A/B on the Radeon 780M
showed that preferring host-visible `DEVICE_LOCAL` for every mapped buffer made
the complete binary segment about **10.65 ms GPU vs 0.0195 ms CPU**, while the
cached mapped policy restored it to **0.137 ms GPU vs 0.015 ms CPU**. The
device-local mapping was therefore rejected; shader-side device locality does
not compensate for slower CPU copies in an H2D/dispatch/D2H hybrid segment.

With the restored policy, the refreshed smoke suite passed every numerical
case. Its four-crop affine-Swish segment was **0.13832 ms GPU vs 0.29722 ms
CPU**, so hybrid selects Vulkan for that shape; the pointwise and residual
pointwise probes remained transfer-bound (**20.6638 vs 0.0269 ms**, and
**14.3158 vs 0.03158 ms**, respectively) and are correctly retained on CPU.
This is the required behavior for `hybrid`: prefer GPU only when the measured
full segment is no slower, while avoiding a silent fallback in `gpu_only`.

Post-change correctness evidence includes small and medium C++/ONNX Runtime
text parity on `en.ppm` (**1/1** each) and a medium mixed-size memory soak over
`16x16`, `mixed`, and the `1920x1080` 151-region UI image. Its post-warm-up
private-byte baseline was **361,832,448** and the next cycle was
**-7,499,776 bytes** from that baseline, consistent with a bounded allocator
high-water rather than retained dynamic workspaces. A six-page tiny batch
regression still returned byte-identical nested results; the shared-host run
measured **126.196 ms serial / 108.076 ms batch (1.168x)**. Timing varies with
contention, so this documents correctness and a measured throughput win rather
than a portable promise.

### 2026-08-11: batched GPU Conv+Swish fusion

### 2026-08-12: tiled batched Vulkan pointwise convolution

The batched Vulkan `1x1` convolution shader now has a workgroup-tiled path
for `C <= 1024`. One 256-thread workgroup owns four output channels and a
256-pixel spatial tile, cooperatively stages the four immutable `[C]` weight
rows in shared memory, then produces four contiguous FP32 values per lane.
This removes the former per-`vec4` weight streaming in the common NCHW
pointwise case. The same path covers Conv, Conv+ReLU, Conv+Swish, and the
equal-shaped residual variants; odd plane tails still use an exact scalar
write and wider channel counts retain the established generic shader path.

This is a real GPU-kernel and batch-throughput improvement, not an invented
end-to-end GPU win. On the local AMD Radeon 780M, the verified four-crop
`N=4,C=24,M=40,plane=257` smoke measurement improved from the earlier
multi-millisecond pointwise dispatches to **0.16412 ms GPU** for Conv and
**0.15202 ms GPU** for Conv+Add+Swish. CPU SIMD still measured **0.02170 ms**
and **0.09334 ms**, respectively, so the complete H2D/dispatch/D2H admission
rule correctly retains CPU on this UMA host. The smoke suite validates all
normal, ReLU, Swish, residual, and non-vector-aligned outputs before timing.

The rebuilt tiny four-page scheduler check (`en`, `line_en`, `mixed`, `zh`;
batch size 4, recognizer parallelism 1, page parallelism 2) remained output
identical after the shader change. This run measured **128.114 ms serial /
88.105 ms batch (1.454x)** on CPU and **125.542 ms / 77.657 ms (1.617x)**
in hybrid mode. These short shared-host samples are scheduler evidence, not a
replacement for the paired ONNX Runtime table above. `gpu_only` remains
explicitly unavailable until the complete PP-OCRv6 graph is device-resident.

For a post-change whole-pipeline sanity A/B on `en.ppm` (16 CPU workers,
batch size 4, recognizer parallelism 1, three warm-ups and ten timed runs),
hybrid measured **34.387 / 72.977 / 216.534 ms** for tiny/small/medium.
The separately sampled CPU tiny run was **36.125 ms**; due shared-host noise
and partial-graph selection these are not substituted for the longer paired
CPU-versus-ORT results. The relevant GPU proof is the verified tiled primitive
and its conservative per-shape full-transfer admission, not an unsupported
claim that all three models now execute fully on the Radeon GPU.

The Vulkan pointwise kernel now also supports the graph-fused PP-OCR pattern
`1x1 Conv -> Sigmoid -> Mul` as an exact FP32 `1x1 Conv + Swish` operation.
Four equal-width NCHW crops share one submission; learned `[M,C]` weights and
`[M]` bias remain compact and can be retained in the persistent parameter
buffers. This removes the CPU-side full-output Swish traversal when a shape is
admitted to Vulkan, without changing the portable AVX-512/AVX2/NEON fallback.

The smoke test verifies the fused output at every element for a four-crop,
non-vector-aligned `N=4, C=24, M=40, plane=257` workload. On the local Radeon
780M its full synchronous H2D/dispatch/D2H measurement was **12.2472 ms GPU
versus 0.13918 ms CPU**, so `hybrid` correctly retains CPU SIMD for that shape.
This is nevertheless a material batch-capable GPU graph building block: a
device/shape is allowed to select it only after this complete measurement and
numerical validation; it is not represented as an end-to-end GPU OCR win on
the present UMA system. The expanded tiny C++/ONNX Runtime text gate passed
**5/5** (`en`, English line, Chinese line, mixed, Chinese page), and the
six-page batch regression preserved identical OCR results at **123.546 ms
serial / 86.991 ms batch (1.420x)** on this run.

### 2026-08-11: shortcut Swish fusion and broader regression pass

Small and medium recognizers contain an inverted-residual suffix
`BatchNorm -> Swish -> Add(shortcut)`. The loader now recognizes this after
the first fusion sweep and emits one `FusedBatchNormSwishAdd` node. Its CPU
kernel produces the affine, exact logistic Swish and equal-shaped residual Add
in one traversal, reusing the dying BatchNorm input allocation. This removes
the intermediate Swish feature map and the separate Add pass. It is protected
by `PPOCR_DISABLE_BN_SWISH_ADD_FUSION=1` for A/B work; a short medium
`mixed.ppm` CPU check (one warm-up / three runs) measured **242.378 ms** fused
versus **260.514 ms** unfused (**7.0% lower latency**), with expected shared
host variance.

Vulkan now exposes the matching batched `channel-affine + Swish + Add`
primitive. Its smoke test validates every output of four `8192`-float NCHW
crops. On the Radeon 780M, complete transfer timing was **0.14988 ms GPU vs
0.14072 ms CPU**, so hybrid correctly chose CPU for that exact case. The graph
path has the same cached no-slower admission rule: it may use Vulkan only when
the full H2D/dispatch/D2H measurement and FP32 validation win; otherwise it
uses the CPU fused path and does not add GPU overhead.

Current post-change evidence: tiny C++/ORT text parity remained **5/5** over
English, Chinese and mixed samples; small and medium each passed an independent
ORT text check on `en.ppm` (**1/1** each). A six-page varied-content tiny batch
regression remained result-identical at **126.926 ms serial / 96.686 ms batch
(1.313x)**. A medium memory soak spanning `16x16`, mixed text, and a dense
`1920x1080` page finished one cycle **7,614,464 bytes below** its
376,975,360-byte post-warm-up baseline, consistent with bounded workspace reuse.

### 2026-08-12: batched Vulkan HardSwish coverage

The graph-fused detector pattern `1x1 Conv -> HardSigmoid -> Mul` now reaches
the Vulkan pointwise kernel as one exact `Conv+HardSwish` dispatch. The shader
receives the ONNX `alpha` and `beta` as raw FP32 push constants and writes
`x * clamp(alpha*x + beta, 0, 1)` directly to the final NCHW output. It uses
the same four-crop Y-dimension batch mapping, shared-weight tile, odd-plane
tail handling, persistent parameter cache, and complete H2D/dispatch/D2H
admission policy as the other pointwise variants. CPU retains its
AVX-512/AVX2/NEON/scalar exact fallback.

The post-change Radeon 780M smoke suite validated every value of an odd-plane
`N=4, C=24, M=40, plane=257` batch. The full synchronous admission sample was
**0.15912 ms GPU / 0.02794 ms CPU**, so this UMA host correctly keeps that
segment on CPU; no end-to-end GPU OCR speedup is claimed. A seven-image tiny
CPU-versus-hybrid output gate (`en`, mixed, CJK, tiny, wide, low-contrast,
blank) passed **7/7** with identical decoded output. With recognizer batch size
4, recognizer parallelism 1, and page parallelism 2, the seven-image hybrid
batch gate preserved output identity and measured **337.565 ms serial /
234.153 ms page-batch (1.442x)**. This is scheduler throughput evidence on
the local machine, not a general GPU performance promise.

### 2026-08-12: three-size CPU/hybrid and memory revalidation

The current build was retested after the Vulkan HardSwish integration with
the official tiny, small, and medium model pairs. The seven-image C++/ONNX
Runtime text gate passed **7/7 for every model** (English, mixed Latin,
CJK, `16x16`, `3000x80`, low-contrast, and blank). The broader tiny stress
corpus also passed **11/11**, including rotated text, dense blobs, and
`1920x1080`/`2560x1440` screenshots.

On `en.ppm` (2 warm-ups, 10 timed runs, 16 CPU workers, recognizer batch 4,
one recognizer worker), the local CPU/hybrid recheck was:

| Model | CPU mean | Hybrid mean | CPU / Hybrid decoded output |
|---|---:|---:|---|
| tiny | 35.164 ms | 34.684 ms | identical |
| small | 71.442 ms | 71.286 ms | identical |
| medium | 205.222 ms | 225.598 ms | identical |

Hybrid is therefore not described as a universal win: each Vulkan segment is
still admitted independently only when its verified complete transfer,
dispatch, and readback time is no slower than the CPU SIMD equivalent. The
medium mixed-size memory soak (`16x16`, mixed text, dense `1920x1080`; two
cycles) used 367,849,472 private bytes after warm-up and ended at
+4,378,624 bytes, consistent with bounded allocator high-water behavior.

### 2026-08-11: batched residual Conv+Swish GPU fusion

The Vulkan batch kernel now also supports the full graph-proven recognizer
suffix `1x1 Conv -> Add(shortcut) -> Sigmoid -> Mul` as mode 9:
`1x1 Conv + residual Add + exact Swish`. Equal-width OCR crops share its Y
dispatch dimension; the learned `[M,C]` weights and `[M]` bias remain compact
and cacheable, and the shortcut is read once before the final device write.
The CPU fallback adds `PointwiseConvAddSwish`, keeping the Conv+shortcut in
the existing AVX-512/AVX2/NEON fused path and overwriting that single output
allocation with exact Swish. Set
`PPOCR_DISABLE_POINTWISE_ADD_SWISH_FUSION=1` to restore the unfused ONNX
sequence for an A/B comparison.

The extended Vulkan smoke test validates every value of an odd-plane
four-crop `N=4,C=24,M=40,plane=257` batch. On the local Radeon 780M, its full
H2D/dispatch/D2H comparison measured **12.3933 ms GPU vs 0.09822 ms CPU**,
so hybrid correctly rejected it. The currently profiled small PP-OCRv6 export
does not emit this exact four-node suffix, so no end-to-end OCR speed claim is
made for the new matcher yet; it is retained as a verified batch-capable
building block for compatible PP-OCRv6 exports, not an unsupported GPU speed
claim on the present UMA host.

### Tiny / small / medium comparison

The same complete two-line OCR pipeline was also measured with the official
three PP-OCRv6 model sizes on this 16-thread AVX-512-capable host.  Model load
is excluded from steady-state latency. The native and ONNX Runtime commands
were run separately with the same PPM input and matching dictionaries. The
following exact-Erf run is the current repeated baseline; `PPOCR_APPROX_GELU`
was unset for every native measurement.

| Model | Native runs / warmup | Pure C++ mean | ORT mean | Native / ORT | Native FPS | ORT FPS |
|---|---:|---:|---:|---:|---:|---:|
| tiny | 20 / 5 | 33.322 ms | 120.106 ms | 0.277x | 30.010 | 8.326 |
| small | 20 / 5 | 83.639 ms | 184.190 ms | 0.454x | 11.956 | 5.429 |
| medium | 20 / 5 | 281.236 ms | 428.584 ms | 0.656x | 3.556 | 2.333 |

This current default-exact run is a same-host comparison with matching model
files and image. It shows lower C++ latency for tiny, small, and medium
(72.3%, 54.6%, and 34.4% respectively). The commands were run as paired
engine measurements, but Windows scheduling still affects individual samples;
the complete JSON output should be retained when refreshing capacity numbers.

The earlier approximate-GELU deployment experiment is retained below as a
separate, opt-in result rather than being conflated with default ONNX-equivalent
execution:

| Model | Native runs / warmup | Pure C++ mean | ORT mean | Native / ORT | Native FPS | ORT FPS |
|---|---:|---:|---:|---:|---:|---:|
| tiny | 10 / 3 | 34.125 ms | 137.535 ms | 0.248x | 29.304 | 7.271 |
| small | 8 / 3 | 95.049 ms | 226.928 ms | 0.419x | 10.521 | 4.407 |
| medium | 8 / 3 | 257.848 ms | 457.041 ms | 0.564x | 3.878 | 2.188 |

All values are from the same 2026-08-11 host and PPM input. The native run
used the persistent 16-worker executor; ONNX Runtime used
`CPUExecutionProvider` with the matching 16-thread budget. The refreshed
measurements show lower mean latency for all three model sizes on this host.
The table below uses the validated optional approximate-GELU deployment setting;
the default exact-Erf paired comparison is documented above.
The opt-in approximation passed the strict 7-image text corpus and the 11-image
size/diversity stress corpus; retain the exact default for applications that
require strict numerical ONNX equivalence.
The specialized stride-2 3x3 gather path remains material for medium:
disabling it measured 2646.179 ms versus 1923.504 ms when enabled with one
requested worker. These are local measurements, not a portable performance
guarantee.

All three model pairs produce the expected two recognized lines in the native
smoke test. The strict 7-image text-parity and 11-image size/diversity stress
gates below use tiny because their known-text expectations correspond to its
dictionary; they exercise input dimensions from 16x16 to 2560x1440.
The latest default-exact verification also forced the dynamic AVX2 fallback
with `PPOCR_DISABLE_AVX512=1`: tiny/small/medium completed at 43.170/113.518/
408.928 ms in a short 8-run sweep. This confirms that AVX-512 remains a
runtime-selected acceleration, not a binary deployment requirement; ARM/NEON
and scalar paths retain the same portable graph implementation.

### Batched constant-weight MatMul fusion

Recognizer attention projections commonly have a batched activation and one
rank-2 immutable weight tensor. The executor now flattens those independent
batch dimensions into one tall GEMM call, avoiding repeated per-batch GEMM
dispatch and allowing the SIMD/NEON GEMM to use its normal row scheduling over
the whole recognition batch. No weights are repacked or duplicated, and every
output row keeps its original increasing-`K` FP32 accumulation order. This is
enabled by default on x86/AVX-512, x86/AVX2, ARM/NEON, and scalar builds;
`PPOCR_DISABLE_MATMUL_BATCH_FOLD=1` provides an A/B fallback.

On the local 16-worker AVX-512 host, exact-GELU `mixed.ppm` measurements with
three recognized crops, batch size four, recognizer parallelism one, three
warmups and fifteen timed runs showed the direct batch benefit for small:
**137.925 ms mean** with the fold versus **195.572 ms** with the fallback
(29.5% lower latency). A separate default scheduler check (batch four,
parallelism fourteen, 3 warmups / 12 runs) measured **41.663 ms tiny**,
**93.915 ms small**, and **340.519 ms medium** on the same input. These
samples are host-specific and the medium workload remains dominated by
convolution, but they confirm that all three model sizes retain the same
correct batched path. Exact ONNX Runtime text parity remained tiny **5/5**,
small **3/3**, and medium **3/3** after the change.
The same fallback path now fuses equal-shape `1x1 Conv -> Add` residual
projections with an AVX2 four-output register tile rather than materializing
the Conv result before a separate Add. A fresh forced-AVX2 3-warmup/10-run
sanity sweep measured tiny/small/medium at 42.446/122.677/434.793 ms on the
shared host; these are coverage and stability samples rather than a paired
ORT comparison. The strict seven-image text gate passed 7/7 with AVX-512
disabled. ARM/NEON has matching four-output fused residual code; it is selected
only when the target compiler exposes NEON, otherwise scalar remains valid.
For AVX-512, large 1x1 projections now use an eight-output register tile
(with four-output/tail handling retained for smaller shapes). It reuses each
loaded input vector across eight output channels and preserves each output
channel's reduction order. The exact-GELU text gate passed 7/7 and the
11-image size/diversity gate passed after this change; the default exact
three-model table above is the associated repeated benchmark.
A 30-run A/B on the same 16-worker host selected this default for the larger
models: small measured 106.047 ms with the eight-output tile versus 109.533 ms
with `PPOCR_DISABLE_AVX512_POINTWISE8=1`; medium measured 301.179 ms versus
313.123 ms. Individual Windows samples remain scheduling-sensitive, but both
the median and mean favoured the eight-output implementation in that repeated
comparison.
The same eight-output technique is implemented for the already fused
`1x1 Conv+ReLU` path, including its exact scalar tail and runtime ISA guards.
It passes the strict text corpus when enabled, but repeated small/medium
end-to-end A/B samples were mixed (and the 30-run medium sample favoured the
default four-output path: 257.914 ms versus 268.169 ms). It is therefore kept
behind `PPOCR_ENABLE_AVX512_POINTWISE_RELU8=1`, rather than silently trading
stable default latency for a workload-specific result.

### Post-fusion comparison refresh

The latest post-fusion same-host refresh (exact Erf-GELU, 16 requested workers,
5 warmups and 20 measured runs per engine) is recorded below. It includes the
new equal-shape `1x1 Conv -> Add` fusion; operator profiling confirmed fused
residual projections in the medium graph (for example `Conv.9_add`,
`Conv.12_add`, `Conv.20_add`, `Conv.50_add`, and `Conv.53_add`). As with the
earlier table, these are sequential paired measurements on a shared Windows
machine, so they document the current build rather than a portable guarantee.

| Model | Pure C++ mean | ORT mean | C++ / ORT | Pure C++ FPS | ORT FPS |
|---|---:|---:|---:|---:|---:|
| tiny | 43.072 ms | 182.025 ms | 0.237x | 23.217 | 5.494 |
| small | 109.535 ms | 244.420 ms | 0.448x | 9.130 | 4.091 |
| medium | 359.697 ms | 642.622 ms | 0.560x | 2.780 | 1.556 |

This refresh measured native latency lower by 76.3%, 55.2%, and 44.0% for
tiny, small, and medium respectively. The observed C++ ranges were
40.202--48.375 ms, 99.140--127.691 ms, and 284.395--396.259 ms; ORT also
showed large host-scheduling variance, especially medium (427.880--1285.338
ms). Keep the original repeated table above for historical comparison and
rerun the paired procedure on the deployment CPU before capacity planning.

The subsequent gate-fusion validation found 13/13, 13/13, and 5/5 fused
HardSigmoid gates in tiny, small, and medium profile runs respectively. Default
and forced-AVX2 strict text parity both passed 7/7, while the 11-image stress
set still passed 11/11. A short shared-host 3-warmup/10-run sanity sweep was
43.159 ms (tiny), 100.075 ms (small), and 416.331 ms (medium); the wide medium
p95 range reflects host scheduling, so these values are deliberately not
presented as a new ONNX Runtime comparison.

The executor now additionally treats a last-use standalone `HardSigmoid` as
an in-place operation. This applies the same exact AVX-512/AVX2/NEON SIMD gate
without allocating a second activation tensor; it is especially relevant to
dynamic image sizes where keeping the graph frontier small lowers transient
memory. The current strict text and 11-image stress checks passed after this
lifetime optimization.
The detector reconstruction head now folds its terminal `ConvTranspose(2x2,
stride 2) -> Add([1,C,1,1])` pairs into the transposed-convolution bias during
model load. This removes both final feature-map Add traversals for tiny, small,
and medium (confirmed as `ConvTranspose.0_bias` and `ConvTranspose.2_bias` in
the medium operator profile) while retaining exact output and all ISA paths.
The strict seven-image comparison and 11-image dimension/diversity stress
suite remained fully passing after the change.

The recognizer's direct `Conv -> Add([1,C,1,1]) -> ReLU` exports are now folded
at load time into the existing Conv+ReLU kernels. This adds the previously
missing no-Identity form of channel-bias folding, so small/medium stem
projections avoid a broadcast Add pass and its transient result. The same
matcher also recognizes canonical `Conv -> HardSigmoid -> Mul` gates and maps
them to the existing exact AVX-512/AVX2/NEON HardSwish kernel when encountered.
The default and forced-AVX2 tiny text-parity checks both passed 5/5 images, and
the 11-image 16x16-to-2560x1440 stress suite passed 11/11 after the change.

### Current optimization validation and comparison refresh

The current build adds the post-Conv Swish graph fusion described above. The
official medium model profiled here does not contain a matching
`Conv -> Sigmoid -> Mul` sequence after affine folding, so this generic fusion
does not contribute to the following current-model timing. The strict tiny
text-parity suite completed 5/5 images against ONNX Runtime after the change.
The benchmark below was then run on the same shared Windows host with
`PPOCR_BENCH_THREADS=16`, AVX-512 enabled dynamically, exact-Erf GELU, five
warmups and 20 measured iterations. It is recorded here as a reproducible
current-build comparison, not a portable capacity guarantee.

| Model | Pure C++ mean | ORT mean | C++ / ORT | Pure C++ FPS | ORT FPS |
|---|---:|---:|---:|---:|---:|
| tiny | 40.159 ms | 140.930 ms | 0.285x | 24.901 | 7.096 |
| small | 95.946 ms | 234.128 ms | 0.410x | 10.422 | 4.271 |
| medium | 342.556 ms | 599.235 ms | 0.572x | 2.919 | 1.669 |

For this run, native C++ latency was 71.5%, 59.0%, and 42.8% lower than ONNX
Runtime CPUExecutionProvider for tiny, small, and medium. Native throughput
was correspondingly 251.0%, 144.0%, and 74.9% higher. The native/ORT
median/p95 pairs were 39.217/44.732 vs. 142.425/199.114 ms (tiny),
94.667/103.429 vs. 234.956/276.820 ms (small), and 316.130/451.931 vs.
576.425/767.443 ms (medium). The shared Windows host is scheduling-sensitive,
so retain the ranges and rerun this paired procedure on a deployment CPU before
capacity planning. The existing tiny/small/medium repeated exact table remains
the longer-run model-size baseline.
### Dense-text stress result

The small-image result above is not representative of a screen with hundreds
of detected text regions. The native path groups equally sized crops into
bounded batches and schedules up to fourteen independent batches concurrently;
each batch retains its output order and uses an isolated execution workspace.
When multiple batches concurrently reach a CPU-parallel operator, one uses the
persistent worker set while contenders execute their independent channel split
locally. This prevents shared-pool serialization and nested-worker contention.
For the PP-OCR recognizer's terminal CTC head it also reduces logits directly
to argmax plus selected-character probabilities, avoiding an otherwise unused
`[batch,time,6906]` Softmax tensor.
Last-use elementwise activations (GELU, BatchNorm+GELU, ReLU, h-swish and
scale/shift) are updated in-place, eliminating transient activation copies. The
BatchNorm+GELU path now evaluates the exact-Erf GELU in the same pass as its
affine transform, removing a further full activation read/write in the default
exact mode. In the explicitly enabled approximate-GELU deployment mode, the
same affine result routes through AVX-512/AVX2 scale-shift plus vector GELU;
the corpus gates above validate that optional path.
Recognizer `Conv -> BatchNorm` inference pairs are folded into immutable
convolution weights and biases during model loading, removing the BatchNorm
activation pass without changing the deployed C++ runtime dependencies.
The detector FPN `2x2`, stride-2 transposed-convolution blocks have dedicated
AVX2 and AVX-512 implementations selected through the existing runtime ISA
dispatch. They vectorize independent input pixels while keeping the same
output/input-channel accumulation order; scalar and ARM builds retain the
portable implementation. In a direct dense-page operator profile, the two
blocks totalled 3.135 ms with AVX versus 3.597 ms scalar (about 12.8% for
those kernels).
The neighbouring detector reconstruction `2x2` valid convolutions now also
have dedicated AVX-512 and AVX2 kernels. They stream the four contiguous input
rows required by each output vector and avoid generic padding/interior checks;
ARM and scalar builds retain the portable kernel. On the current host this
changed the complete two-line small model from 110.364 ms to 98.112 ms and the
medium model from 352.690 ms to 338.791 ms in the same short-run configuration
(about 11.1% and 3.9% respectively), while dense tiny recognition measured
243.949 ms for 123 boxes.
The larger stride-one `3x3` detector layers use an AVX-512 four-output-channel
kernel as well. Each input vector is loaded once and accumulated into four
independent channel outputs, retaining the original channel/kernel accumulation
order. On the medium profile this reduced `Conv.121` from 26.809 ms to 5.910 ms
and `Conv.3` from 20.759 ms to 12.379 ms; the complete short-run medium OCR
measurement improved from 338.791 ms to 291.049 ms. AVX2, NEON, and scalar
paths remain available through runtime ISA dispatch.
The same four-output `3x3` algorithm is now implemented for AVX2 as well. An
explicit `PPOCR_DISABLE_AVX512=1` run exercised that fallback on the present
machine: medium completed in 404.166 ms and the strict tiny text corpus passed
7/7, confirming that advanced instructions remain dynamically optional rather
than a deployment requirement. ARM/NEON retains its portable vector path.
The scheduler now divides these fused `3x3` layers by four-channel groups,
instead of splitting a group between CPU workers. This preserves input-vector
reuse even for channel-parallel medium layers and safely sends only an incomplete
last group to the scalar tail. The resulting AVX-512 medium run measured
279.329 ms (3.580 FPS); strict text and the 11-image dimension/diversity corpus
both remained fully passing.
The same AVX-512 four-output strategy now covers the medium detector's square
stride-one `5x5` and `7x7` context layers. It cut the profiled `Conv.111`
7x7 from 8.472 ms to 3.898 ms and `Conv.114` 5x5 from 3.973 ms to 1.938 ms;
the complete medium result improved from 279.329 ms to 257.848 ms (3.878 FPS).
Unsupported ISAs and non-square/asymmetric kernels continue through the
existing AVX2/NEON/scalar paths.
Depthwise dispatch now routes square stride-one `7x7` and `9x9` kernels to the
existing AVX-512/AVX2 vector kernel too. This covers medium `Conv.64` (256
channels, 9x9): its local operator profile fell from 8.080 ms to 4.845 ms.
The native benchmark's current medium distribution remains sensitive to shared
host scheduling, so the paired end-to-end table is retained as the comparable
model-size result rather than replacing it with an unpaired one-off sample.
For `1x1 Conv -> ReLU` sequences, the AVX-512 path now applies the clamp before
storing each register-tiled output vector. This removes the separate activation
read/write for the fused detector projections while leaving AVX2, NEON, and
scalar execution on the existing safe Conv-then-ReLU path. The strict text and
11-image dimension/diversity gates passed after the change. On this shared
benchmark host the short medium samples remained scheduler-noisy, so no new
end-to-end speed claim is made without a paired repeated run.
The default exact-Erf GELU path is also parallelized across independent NCHW
planes through the persistent executor. BatchNorm+GELU now precomputes its
per-channel affine coefficients once and submits plane-local exact evaluation
without allocating another activation. On medium profiling this changed the
13 fused GELU calls from 26.743 ms to 7.297 ms while retaining default ONNX
Erf semantics; the strict text and 11-image stress gates passed with
`PPOCR_APPROX_GELU` unset. The optional approximate mode remains available for
deployment-specific validation, not as a correctness requirement.
The recognizer terminal CTC Top-1 reduction now parallelizes only across
independent batch sequences. Each sequence retains its own blank/repeated-token
state, while dense-page batches can distribute vocabulary scans and the selected
character confidence softmax over the persistent worker set. Exact decoded-text
and the 11-image dimension/diversity gates passed in default exact-GELU mode;
the shared-host dense benchmark remains variable, so its paired table remains
the authoritative capacity-planning reference.
CTC's 6,906-way per-step ArgMax also uses dynamically selected AVX-512 or AVX2
vector reductions. It explicitly scans the non-vector-width tail before
locating the first winning class, preserving the scalar tie rule and ensuring
the final vocabulary entries (including the space class) remain observable.
Both default AVX-512 and forced `PPOCR_DISABLE_AVX512=1` AVX2 runs passed the
strict 7-image text corpus; the current exact-GELU dense 123-box run measured
345.415 ms (median 343.977 ms, p95 359.279 ms), subject to the documented
shared-host scheduling variation.
AVX-512 four-output fusion now also covers stride-one asymmetric `1x3/1x5/1x7`
and `3x1/5x1/7x1` detector convolutions. It vectorizes the contiguous spatial
axis, preserves scalar border accumulation, and assigns four-channel groups to
workers so input loads are reused. Medium `Conv.112` (7x1) fell from a recent
3.363 ms profile to 1.211 ms. Default exact-GELU strict-text and 11-image
dimension/diversity gates remained passing; standalone medium timing is still
reported with the documented shared-host variance rather than as a new paired
ORT claim.
The DB connected-component postprocessor represents threshold and visited state
in one byte per probability-map pixel (`0` background, `1` pending, `2` visited),
instead of two full byte maps. This halves that postprocessor workspace while
leaving thresholding, traversal order, scores, boxes, and reading order unchanged.
The detector terminal `Sigmoid` is also executed in-place whenever its input
has no remaining consumers. It reuses the final DB-logit allocation as the
probability map, eliminating a peak activation allocation and copy while
preserving the exact scalar sigmoid expression.
Detector FPN nearest-neighbour integer upscales are implemented as direct
NCHW row replication, and recognizer inverse-Transpose pairs plus Conv channel
bias broadcasts are removed at model load. These changes eliminate dynamic
activation copies without relaxing the exact-GELU accuracy path.
The executor also prunes original constants made unreachable by load-time
Conv/BatchNorm folding. This applies to tiny, small, and medium without
changing runtime graph inputs. A fresh two-cycle mixed-size medium soak
(16x16, 3000x80, noisy scene and 1920x1080 UI page) used a 146.94 MiB
post-warmup baseline; the cycles were 147.21 and 148.18 MiB (+0.27 and
+1.24 MiB). The variation is Windows heap retention rather than monotonic
per-shape growth; longer soak runs should be used for release qualification.
The same mixed-size medium sequence in default exact-GELU mode used a 147.68
MiB post-warmup baseline; two cycles ended at 146.48 MiB and 152.16 MiB
(-1.20 / +4.48 MiB). This is bounded allocator retention rather than a
per-shape activation cache; use longer production soak runs for qualification.
A fresh four-cycle medium run after the pointwise-tile update used a 149.641
MiB post-warmup baseline and ended at 146.504 MiB (-3.137 MiB); its largest
cycle was only 2.022 MiB above baseline. The mixed sequence covered 16x16,
3000x80, low-contrast, and 1920x1080 dense-page inputs, again showing bounded
Windows allocator retention rather than persistent dynamic-shape growth.
The AVX2/AVX-512 direct-convolution path now also covers stride-1 7x7 and
asymmetric 3/5/7x1 or 1x3/5/7 detector kernels, while preserving the same
output/input-channel/kernel accumulation order; ARM and scalar paths retain
the portable implementation.
The dominant NCHW `1x1` convolution kernel also now computes four output
channels per shared SIMD input load (AVX-512, AVX2, and NEON implementations),
reducing hot-path input bandwidth while preserving each channel's accumulation
order.
The detector DB-head `2x2`, stride-1 `SAME_UPPER` max pool is likewise mapped
to a boundary-safe AVX-512/AVX2/NEON kernel, removing generic window and bounds
reconstruction from every output pixel.
The following repeatable stress measurement records the present result on
`build/ui_dense_1920x1080.ppm` (about 123--124 text regions):

| Engine | Warmup | Measured runs | Mean latency | Throughput |
|---|---:|---:|---:|---:|
| Pure C++ PP-OCRv6 | 4 | 10 | 233.805 ms | 4.277 FPS |
| ONNX Runtime 1.28 CPUExecutionProvider | 4 | 10 | 929.357 ms | 1.076 FPS |

The table uses one paired `compare.ps1` run with the persistent executor and
validated approximate-GELU setting. C++ is **74.8% lower latency** (0.252x) and
**297.5% higher throughput** in this specific run. It replaces the previous
serial-recognition baseline of 2680.808 ms, a **11.47x** native speedup.
A repeated parameter
sweep after the copy-elimination changes selected the current default of
`rec_batch_size = 4` and
`rec_parallelism = 14` for this 16-logical-CPU host; applications can tune
these options for their deployment CPU. Short dense runs can vary
materially with CPU scheduling, so use the command below and more runs on the
deployment target before capacity planning:

`Options::rec_batch_width_bucket` (or benchmark environment variable
`PPOCR_REC_WIDTH_BUCKET`) can additionally pack nearby natural crop widths
into one zero-right-padded batch; each crop keeps its own aspect-ratio resize.
The default remains `1`: a local 123-crop dense-page sweep did not demonstrate
a stable CPU benefit from padded work, so exact-width batching remains the
lower-memory default. Wider buckets are available for deployment-specific GPU
or batch-density tuning.

```powershell
powershell -ExecutionPolicy Bypass -File .\bench\compare.ps1 `
  -Image .\build\ui_dense_1920x1080.ppm -Runs 10 -Warmup 4 -Threads 16
```

The captured result is C++ 233.805 ms (median 232.980 ms, p95 244.485 ms) and
ONNX Runtime 929.357 ms (median 930.224 ms, p95 1098.989 ms). Keep repeated
paired measurement when refreshing the table, because dense-page latency can
vary with host scheduling.

The run used four warmups, 10 measured runs, and matching 16-thread CPU budgets.
Use longer repeated measurements on the deployment CPU before capacity planning;
dense-page latency also depends strongly on the detected crop mix.


### Expanded stress corpus

`bench/stress_regression.py` executes the native C++ pipeline over 11 deterministic
local stress images without adding a runtime dependency. It covers 16鑴?6 tiny text,
3000鑴?0 and 80鑴?000 strips, low contrast, rotation, blank input, dense synthetic
text, and 1920鑴?080 / 2560鑴?440 screenshots. The gate asserts bounded detection
counts, valid confidence/geometry, and high-confidence text anchors where exact
front-end parity is meaningful; rotated and density-only cases intentionally test
robustness rather than unstable line ordering.

```powershell
& 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' `
  .\bench\stress_regression.py `
  --cpp .\build\ppocr_demo.exe `
  --det D:\workprj\aicoder\.tmp\ocr-models\ppocrv6_tiny_det.onnx `
  --rec D:\workprj\aicoder\.tmp\ocr-models\ppocrv6_tiny_rec.onnx `
  --dict D:\workprj\aicoder\.tmp\ocr-models\dict_ppocrv6_tiny.txt `
  --images D:\workprj\aicoder\corelib\ocr\testdata\stress
```

Latest local run: **11/11 passed** (box counts: 123, 0, 1, 1, 20, 1, 3, 6, 743,
34, 38). This broadens the existing strict 7-image text-parity gate; it does not
weaken it.

### Recognition-regression corpus

`bench/accuracy_compare.py` compares complete C++ OCR results against an ONNX
Runtime CPU reference. Its default gate checks detected-line count and decoded
text; confidence and geometry are retained in the JSON report but do not gate
the test because the two standalone frontends have small resize/crop rounding
differences. Pass `--strict-geometry` to also gate those fields.

`bench/accuracy_expectations.json` supplies independently recorded expected
text for the deterministic corpus, so the same run also checks C++ results
against known text rather than only comparing two inference backends.

The latest 7-image corpus passed **7/7 text-parity cases**. It includes clean
multi-line English, a single word, a 3000-pixel-wide receipt line, a
low-contrast/noisy scene, a blank image, and a 16x16 tiny-text case. The
samples are existing deterministic test assets from
`D:\workprj\aicoder\corelib\ocr\testdata`.

An additional multilingual and extreme-shape sweep covers Chinese and mixed
language text, a 80x3000 vertical strip, rotated text, synthetic dense blobs,
and 1920x1080/2560x1440 screenshots. Chinese/mixed baseline samples retain
exact decoded-text parity. The pathological vertical/dense screenshot samples
remain diagnostic rather than release gates because detector/crop rounding can
alter weak low-confidence regions between the two independently implemented
frontends. `accuracy_compare.py` now forces UTF-8 report output so this corpus
can be run from a Windows GBK console without losing dictionary characters.

```powershell
$images = @(
  'D:\workprj\aicoder\corelib\ocr\testdata\en.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\line_en.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\stress\single_word.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\stress\wide_strip_3000x80.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\stress\noisy_lowcontrast.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\stress\blank_800x600.png',
  'D:\workprj\aicoder\corelib\ocr\testdata\stress\tiny_16x16.png'
)
& 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' `
  .\bench\accuracy_compare.py `
  --cpp .\build\ppocr_demo.exe `
  --python 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' `
  --det 'D:\workprj\aicoder\.tmp\ocr-models\ppocrv6_tiny_det.onnx' `
  --rec 'D:\workprj\aicoder\.tmp\ocr-models\ppocrv6_tiny_rec.onnx' `
  --dict 'D:\workprj\aicoder\.tmp\ocr-models\dict_ppocrv6_tiny.txt' `
  --expectations .\bench\accuracy_expectations.json `
  --images $images
```

Captured report: `build/accuracy_ground_truth_7_ctctop1_sparse.json` (7/7
passed after batch/concurrent recognition and terminal-CTC reduction).

### Dynamic-size memory soak

`ppocr_mem_soak` keeps one C++ OCR instance alive while repeatedly alternating
tiny, wide-strip, noisy, and 1920x1080 UI images. It reports process-private
bytes after every complete mixed-size cycle. Intermediate tensors are released
after their final graph consumer, model weights remain immutable, and no
per-shape activation cache is retained. Detector results (including the large
probability map) are additionally scoped through DB post-processing and released
before concurrent recognition batches begin, so the allocator can reuse their
storage rather than combining detector and recognizer activation peaks.

The same process-level workload can be run against ONNX Runtime with
`bench/ort_mem_soak.py`. On this Windows host, using 12 mixed-size cycles:

| Engine | Baseline private bytes | Final private bytes | Final change |
|---|---:|---:|---:|
| Pure C++ PP-OCRv6 | 23.825 MiB | 24.633 MiB | +0.809 MiB |
| ONNX Runtime CPU | 145.371 MiB | 203.805 MiB | +58.434 MiB |

The current C++ run ranged from 22.735 MiB to 27.703 MiB after the baseline;
its final working set was 0.809 MiB above the immediate post-load baseline,
without a monotonic per-cycle increase. ONNX Runtime retained roughly 58 MiB
in the prior corresponding run. This is a process-private-memory measurement
for the specified machine and models, not a portable memory guarantee; rerun
it on the deployment target.

Build PPM copies for the native soak from the same PNG source files:

```powershell
$code = @'
from PIL import Image
from pathlib import Path
source = Path(r'D:\workprj\aicoder\corelib\ocr\testdata')
output = Path(r'.\build\mem_samples'); output.mkdir(exist_ok=True)
for name in ['stress/tiny_16x16.png', 'stress/wide_strip_3000x80.png',
             'stress/noisy_lowcontrast.png', 'stress/ui_dense_1920x1080.png']:
    Image.open(source / name).convert('RGB').save(output / (Path(name).stem + '.ppm'), format='PPM')
'@
$code | & 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' -
```

Then run the C++ side (replace the model paths if needed):

```powershell
.\build\ppocr_mem_soak.exe <det.onnx> <rec.onnx> <dict.txt> 12 `
  .\build\mem_samples\tiny_16x16.ppm `
  .\build\mem_samples\wide_strip_3000x80.ppm `
  .\build\mem_samples\noisy_lowcontrast.ppm `
  .\build\mem_samples\ui_dense_1920x1080.ppm
```

### Resolution ladder and post-large-image memory recovery

For a stricter retention check, generate a deterministic page ladder that
climbs from `16x16` through `3840x2160`, includes ultra-wide and ultra-tall
pages, and ends with another `64x64` page:

```powershell
& 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' `
  .\bench\generate_memory_ladder.py --output .\build\memory_ladder

.\build\ppocr_mem_soak.exe <det.onnx> <rec.onnx> <dict.txt> 2 `
  .\build\memory_ladder\ladder_16x16.ppm `
  .\build\memory_ladder\ladder_64x64.ppm `
  .\build\memory_ladder\ladder_160x120.ppm `
  .\build\memory_ladder\ladder_320x240.ppm `
  .\build\memory_ladder\ladder_640x480.ppm `
  .\build\memory_ladder\ladder_1280x720.ppm `
  .\build\memory_ladder\ladder_4096x256.ppm `
  .\build\memory_ladder\ladder_256x4096.ppm `
  .\build\memory_ladder\ladder_1920x1080.ppm `
  .\build\memory_ladder\ladder_2560x1440.ppm `
  .\build\memory_ladder\ladder_3000x1600.ppm `
  .\build\memory_ladder\ladder_3840x2160.ppm `
  .\build\memory_ladder\ladder_recovery_1920x1080.ppm `
  .\build\memory_ladder\ladder_recovery_4096x256.ppm `
  .\build\memory_ladder\ladder_recovery_256x4096.ppm `
  .\build\memory_ladder\ladder_return_64x64.ppm
```

For CI or a repeatable pass/fail report, run the bounded-recovery checker. It
records every resolution and requires the final `64x64` sample to remain
within 20% of both the warmed baseline and the final ultra-tall recovery
sample, as well as within 20% of the earlier `3840x2160` sample. The report
includes `declined_after_largest`, so allocator capacity retention
cannot be confused with an actual post-large-image recovery observation:

```powershell
& 'D:\workprj\aicoder\.tmp\ocr-models\.venv\Scripts\python.exe' `
  .\bench\memory_ladder_regression.py `
  --cpp .\build\ppocr_mem_soak.exe --det <det.onnx> --rec <rec.onnx> `
  --dict <dict.txt> --images .\build\memory_ladder --cycles 2 --backend cpu `
  --require-post-large-decline
```

`ppocr_mem_soak` loads, infers, and destroys one RGB image at a time and emits
one private-byte sample after every resolution.  The final small-page sample
is therefore the direct recovery observation after the staged 4K and
extreme-aspect-ratio tail; compare it with the `ladder_3840x2160` and final
`ladder_recovery_256x4096` samples and the next cycle rather than treating
the process allocator's high-water capacity as a leak.  Run the same command
with `PPOCR_BACKEND=hybrid` to include the persistent Vulkan buffers in the
process-level measurement.

The current generator intentionally uses an ascending, aspect-ratio-diverse
16-sample sequence: `16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 ->
1280x720 -> 4096x256 -> 256x4096 -> 1920x1080 -> 2560x1440 -> 3000x1600 ->
3840x2160 -> 1920x1080 -> 4096x256 -> 256x4096 -> 64x64`. The final staged
recovery tail checks that memory remains bounded after both 4K area and
extreme-aspect-ratio images. In addition to private bytes,
the native JSON now includes Windows working-set bytes for each measured
sample.  Private bytes remain the bounded-recovery gate; working set is
reported as complementary residency evidence because the Windows heap and
Vulkan driver may retain reusable committed capacity after the page and its
activations have been destroyed.

The generator also includes native **5K** (`5120x2880`) and **8K**
(`7680x4320`) source pages.  The 8K sample is followed immediately by a 4K
recovery sample, so use `--require-post-large-decline` to make both Windows
private bytes and working set decline at that exact transition a strict gate.
This catches source-image retention even though detector inference itself is
normally capped by `det_limit_side_len`.

### 2026-08-14: current small-to-large image-memory check

The checked-in generator was rerun against the current medium PP-OCRv6 CPU
build.  It creates the deterministic 16-image sequence above, processes each
image with one long-lived OCR instance, and destroys the decoded RGB image
before sampling process memory.  The result therefore measures retained
runtime/allocator capacity rather than intentionally retained source images.

In the current one-cycle run, private memory was **147,017,728 B** at
`3840x2160`, then immediately declined to **146,296,832 B** at the following
`1920x1080` sample (**-720,896 B**, **-0.49%**).  Working set also declined
from **150,679,552 B** to **150,085,632 B** (**-593,920 B**).  After the
complete recovery tail, the final `64x64` sample was **147,124,224 B** private
and **151,035,904 B** working-set memory: below the pre-return peak
(**148,320,256 B** private / **152,371,200 B** working set), and only **2.21%**
above the warmed private baseline (**143,937,536 B**).  The bounded-recovery
gate passed.  Full per-resolution evidence is
`build/memory_ladder_medium_cpu_current_20260814.json`.

Windows allocation is intentionally treated as a bounded-recovery result, not
as a requirement that every allocator page be decommitted immediately: the
explicit first-post-4K samples prove the observed downward transition, while
the final small-image sample guards the durable footprint after the whole
large-image sequence.

### 2026-08-13: current ascending-resolution memory measurement

With the current default recognizer batch size of 1, CPU backend, 16 kernel
workers, and approximate GELU, medium PP-OCRv6 passed a fresh one-cycle
ascending-resolution check. The run covers every generated image from
`16x16` through `3840x2160` (including `4096x256` and `256x4096`), then the
staged recovery tail and a final `64x64`. Private bytes were **147,234,816 B**
after `3840x2160` and **147,144,704 B** after the final `64x64`, a decline of
**90,112 B (0.06%)**. Working set likewise declined from **151,109,632 B** to
**150,913,024 B** (**196,608 B**). The final private-byte value was **0.008%**
above the fully warmed **147,132,416 B** baseline and 2.38% below the
pre-return peak, so every 20% bounded-recovery gate passed. The complete
per-resolution record is
`build/memory_ladder_medium_cpu_ascending_recheck_20260813.json`; values are
host-specific allocator observations, rather than a promise of immediate heap
decommit on every operating system.

### 2026-08-13: per-resolution memory-record validation

The ladder checker now preserves the private-byte and working-set sample for
**every** image resolution in its JSON report, together with the cycle peak
and a direct post-`2560x1440` recovery flag.  This makes the check suitable for
both leak detection and deployment profiling: a final small-page allocation is
not mistaken for the large-page allocation just because a heap or driver keeps
reusable capacity committed.

On the current AVX-512 Windows host, medium PP-OCRv6 with 16 kernel workers
and opt-in approximate GELU passed two fresh CPU ladder cycles.  The warmed
private baseline was **146,632,704 bytes**.  After `2560x1440`, the final
`64x64` sample was **148,533,248 bytes** in cycle 1 (+1.30% baseline, +2.00%
from the largest-page sample) and **143,220,736 bytes** in cycle 2 (-2.33%
baseline, -1.23% from the largest-page sample).  Thus direct private-byte
recovery occurred in the second cycle but not the first: Windows heap reuse
can retain or move committed capacity between samples.  Both final samples
remained within the 20% bounded-recovery limit, which is the pass/fail gate;
the JSON deliberately retains the explicit per-cycle recovery flag instead of
claiming unconditional decommit after a large image.

The same two-cycle Hybrid Vulkan run also passed.  Its warmed private baseline
was **394,969,088 bytes** and each final `64x64` sample was **396,017,664
bytes** (+0.265%).  The final private bytes equalled the immediately preceding
`2560x1440` sample in both cycles, while the working set was essentially flat
(+12,288 bytes in cycle 1 and +8,192 bytes in cycle 2).  This is persistent
Vulkan/Windows allocator capacity rather than evidence that an image or its
activations remain live.  These are machine-specific process measurements;
the exact per-resolution values are retained in
`build/memory_ladder_medium_{cpu,hybrid}_20260813.json` when the commands are
run locally.

On 2026-08-12, the current medium model passed this expanded two-cycle gate.
CPU warmed at **148,127,744 B** and returned to **147,148,800 B** and
**147,152,896 B** (both **-0.66%** of baseline); the final small-page samples
were 2.33% and 1.70% above their immediately preceding largest-page samples.
Hybrid Vulkan warmed at **393,490,432 B** and returned at **395,317,248 B**
(+0.46%) and **395,341,824 B** (+0.47%).  All private-byte deltas passed the
20% gate.  The resident working-set observations are also captured in the
checker output rather than being inferred from private bytes; on this shared
Windows host they remained within the allocator/driver's warmed range, not a
claim that either heap must promptly decommit pages.

On 2026-08-12, the tiny model with 16 CPU workers was run through this exact
seven-image ladder.  CPU-only warmed to **9,400,320 bytes**; after the largest
`2560x1440` page its next small-page sample was **9,465,856 bytes**
(**+65,536 bytes**, 0.7% of baseline).  Hybrid warmed to **116,305,920 bytes**
and the corresponding post-large small-page sample was **115,802,112 bytes**
(**-503,808 bytes**).  These per-image values show recovery after the largest
input for both modes.  They are Windows private-byte observations on this
host, not a cross-platform leak proof or a fixed memory guarantee.

The same seven-page gate was rerun after the depthwise-pointwise-GELU graph
fusion with the **medium** PP-OCRv6 model (two measured cycles, default bounded
CPU scheduler).  Both modes passed the 20% bounded-recovery limits:

| Backend | Warmed baseline | Final `64x64`, cycle 1 | Final `64x64`, cycle 2 | Result after `2560x1440` |
|---|---:|---:|---:|---|
| CPU | 136.351 MiB | 140.019 MiB (+2.69%) | 140.008 MiB (+2.68%) | Allocator private bytes did not immediately decline, but remained stable and 2.24閳?.29% above the largest-page sample. |
| Hybrid Vulkan | 360.574 MiB | 361.582 MiB (+0.28%) | 361.582 MiB (+0.28%) | Equal to the immediately preceding largest-page sample in both cycles. |

The CPU observation is expected on Windows: completed image and activation
objects have been destroyed, while the process allocator may retain reusable
blocks.  Consequently the test reports both `declined_after_largest` and the
bounded deltas instead of treating an unchanged private-byte high-water mark
as proof of a live-image or tensor leak.  The Hybrid samples include its
persistent Vulkan allocations.

The same CPU-only one-cycle ladder was also run for the other two PP-OCRv6
sizes: small warmed to **37,662,720 bytes** and returned at
**38,973,440 bytes** after its largest page (**+1,310,720 bytes**); medium
warmed to **147,435,520 bytes** and returned at **148,439,040 bytes**
(**+1,003,520 bytes**).  In both cases the return sample remained within 3.5%
and 0.7% of their post-warm-up baselines respectively, with no retained RGB
source images between calls.

The reusable checker was then run for two full ladders after the per-image
sampler change.  All three CPU models passed the 20% bounded-recovery gate:
tiny returned within **+1.14%**, small within **+10.09%**, and medium within
**+2.46%** of their warmed baselines; each final `64x64` sample was also at or
below the pre-large `1920x1080` sample except medium's first cycle (**+0.81%**).
Tiny hybrid also passed, returning within **+0.08%** of baseline, including
the persistent Vulkan allocation.  These figures are deliberately a bounded
allocator-reuse test, not an assertion that Windows must immediately release
committed heap pages to the OS.

The current checker also makes the immediate recovery result explicit.  On a
fresh two-cycle tiny run with the default 12 workers, CPU warmed to
**10,047,488 bytes** and returned at **9,379,840** then **9,441,280 bytes**;
both cycles reported `declined_after_largest=true` (2.97% and 9.64% below the
preceding `2560x1440` samples).  Hybrid warmed to **135,745,536 bytes** and
returned at **135,098,368** then **135,102,464 bytes**, both below its warmed
baseline but **not** immediately below the preceding largest-image samples
(`declined_after_largest=false`, +1.24% and +0.63%).  This distinction is
intentional: the test reports the actual post-large behavior and separately
enforces bounded recovery, rather than incorrectly claiming immediate heap
release for persistent Vulkan/Windows allocator capacity.

### 2026-08-12: current ascending-resolution memory check

The deterministic seven-image ladder was regenerated and rerun on the current
binary: `64x64 -> 320x240 -> 640x480 -> 1280x720 -> 1920x1080 -> 2560x1440
-> 64x64`.  Each sample performs `load -> Recognize -> destroy ->
PrivateBytes`, so the reported process-private bytes include runtime working
memory but do not retain the source RGB page between samples.  The bounded
20% return gate passed in every run below.

| Model / backend | Warmed baseline | Cycle 1 final 64x64 | Cycle 2 final 64x64 | Post-largest result |
|---|---:|---:|---:|---|
| tiny / CPU | 9,367,552 B | 9,412,608 B (+0.48%) | 9,424,896 B (+0.61%) | Fell below the preceding 2560x1440 sample in both cycles (11.17%, 1.75%). |
| medium / CPU | 148,254,720 B | 146,284,544 B (-1.33%) | 147,300,352 B (-0.64%) | Bounded return; final sample was 1.08% / 2.65% above the immediately preceding large sample. |
| tiny / hybrid Vulkan | 115,851,264 B | 99,143,680 B (-14.42%) | 98,140,160 B (-15.29%) | Bounded return; final sample was 0.004% / 0.017% above the immediately preceding large sample. |

`declined_after_largest` is therefore true for the CPU tiny run and false for
the medium CPU and hybrid runs.  That false value is an observation, not a
leak verdict: Windows heap and persistent Vulkan allocations may retain or
rearrange reusable capacity.  The two complementary checks閳ユ敄he final-small
sample versus its post-warm-up baseline and the stated largest-page
comparison閳ユ攷ake the memory behavior auditable without claiming that a process
must immediately return committed pages to the OS.

After the subsequent depthwise-pointwise-GELU fusion, the medium ladder was
rerun with two cycles.  CPU warmed at **142,974,976 B**; its final `64x64`
samples were **146,821,120 B** (+2.69%) and **146,808,832 B** (+2.68%).
They were 2.29% and 2.24% above the respective `2560x1440` samples, so
`declined_after_largest=false`, yet both are stable and well within the 20%
bounded-return gate.  Hybrid warmed at **378,089,472 B** and returned at
**379,146,240 B** in both cycles (+0.28%); each final small image equalled its
preceding largest-image sample (`declined_after_largest=true`).  Thus the
test continues to distinguish released OCR image/tensor lifetimes from normal
Windows heap or persistent Vulkan capacity retention.

### 2026-08-12: admitted depthwise-to-pointwise Vulkan segment

The graph loader now recognizes an exact `depthwise Conv(bias) -> 1x1
pointwise Conv(bias)` pair and emits `FusedDepthwisePointwiseConv`.  Its CPU
path remains the same two SIMD kernels, while hybrid can submit the existing
Vulkan chain in one command buffer: the depthwise result stays in a device
storage buffer and only the chain boundaries transfer.  Admission is cached
per full NCHW geometry and accepts GPU only after a repeated H2D + dispatch +
fence + D2H comparison and elementwise CPU-result validation show that the
complete segment is no slower.  Consequently this reduces CPU occupancy when
the Radeon/driver wins without forcing a transfer-bound GPU route.

The feature covers the detector MobileNet backbone pairs in all three model
sizes.  AVX-512 kernel smoke and the four diverse decoded-text comparisons
for tiny, small and medium all passed (**12/12**).  The current Radeon 780M
primitive probe measures a representative depthwise-to-pointwise batch at
**0.76714 ms GPU vs 0.20474 ms CPU**, so this particular UMA shape is
correctly rejected; it is evidence that the new graph route retains the
no-slower policy rather than a claim of blanket GPU acceleration.  A current
20-run CPU-only `en.ppm` snapshot (16 kernel workers, approximate GELU)
measured tiny **27.954 ms**, small **60.455 ms**, and medium **195.588 ms**.
`gpu_only` still fails explicitly because the complete PP-OCRv6 graph,
post-processing and CTC decode are not yet device resident; it never silently
falls back to CPU.

### 2026-08-12: depthwise-to-pointwise CPU graph fusion

The same strict MobileNet `depthwise Conv(bias) -> 1x1 pointwise Conv(bias)`
pattern now executes as one graph node on CPU too.  It removes the executor's
hash-map entry, node dispatch and intermediate tensor lifetime from the graph;
the internal temporary remains only for the two exact SIMD kernels.  This is
an intentional graph/lifetime fusion, not a mathematical approximation.  Set
`PPOCR_DISABLE_DEPTHWISE_POINTWISE_FUSION=1` for an A/B fallback.

On the current 16-worker AVX-512 host, three warm-ups and fifteen `en.ppm`
runs showed the fused path at **26.784 / 56.596 / 174.962 ms** for tiny,
small and medium, respectively, compared with **27.121 / 58.209 / 228.572
ms** with the graph fusion disabled.  The medium result is host-scheduling
sensitive, so the reproducible conclusion is the consistent no-regression
and gains for tiny/small, while deployment owners should rerun the medium A/B
on their target CPU. A separate 20-run medium A/B measured **213.711 ms**
fused versus **222.719 ms** disabled.  The exact decoded-text gate passed
**12/12** across tiny/small/medium and four diverse inputs.  A two-cycle
medium resolution ladder also passed the bounded memory-recovery gate:
147,255,296-byte warmed baseline; final small-page samples were -0.68% and
-0.08% from baseline.  As elsewhere, the final samples did not immediately
fall below the preceding maximum-size sample, which is normal reusable
Windows allocator capacity rather than a leak assertion.

### 2026-08-12: depthwise-pointwise-GELU liveness fusion

The loader now extends that exact single-use pattern through an immediately
following fused GELU as `FusedDepthwisePointwiseConvGelu`.  The pointwise
output is activated in place, rather than re-entering the generic graph loop
to rename and transform it.  It preserves FP32 operation order (the same
existing GELU kernel runs after the two convolutions) and keeps the Vulkan
segment's no-slower admission boundary intact; a selected GPU segment still
returns before the exact CPU GELU pass.  The targeted A/B escape hatch is
`PPOCR_DISABLE_DEPTHWISE_POINTWISE_GELU_FUSION=1`.

The AVX-512 kernel smoke and the 12/12 decoded-text ONNX Runtime gate passed.
On this shared host, a 15-run `en.ppm` A/B did not establish a uniform
latency benefit: tiny was 29.137 vs 28.908 ms, small 62.292 vs 62.007 ms,
and medium 205.199 vs 221.369 ms (fusion enabled vs disabled).  It is retained
because it removes a graph-lifetime boundary and shows a medium improvement,
but the table intentionally does not promote it to a universal CPU speedup.

### 2026-08-12: expanded batched ordinary-Conv SIMD coverage

`Conv2dBatch()` now carries the batch index through the existing runtime-gated
SIMD kernels for valid `2x2`, padded `5x5`/`7x7`, and asymmetric
`1x3`/`3x1`/`1x5`/`5x1` stride-one convolutions, in addition to the prior
`3x3` stride-one/two paths.  This removes one worker-pool dispatch per crop
for the medium detector's context layers while preserving each output's
increasing-channel/kernel accumulation order.  AVX-512 is selected only after
the existing CPUID/XCR0 guard; AVX2 and portable/NEON paths retain their safe
fallback behavior.

The low-level smoke now compares N=3 batched output against three independent
calls for all of those shapes and ReLU tails.  It passed with normal AVX-512
dispatch and with `PPOCR_DISABLE_AVX512=1` (AVX2 fallback).  A post-change
four-identical-page medium CPU batch test (`det_batch_size=4`,
`rec_batch_size=4`, three measured runs) was result-identical and measured
**793.299 ms** serial versus **710.437 ms** batched (**1.117x**).  The same
tiny hybrid configuration was result-identical and measured **126.750 ms**
serial versus **85.669 ms** batched (**1.480x**).  These are local scheduler
measurements, not a universal Vulkan claim: the concurrent Vulkan primitive
smoke continued to select GPU only for its no-slower affine+Swish probe and
retained CPU for the transfer-bound Conv and GEMM probes.

The full small-to-large-to-small CPU ladder was rerun for all model sizes with
the expanded kernels.  All passed the three 20% bounded-recovery limits.
Observed final-small change from the post-warm baseline was tiny **+1.72%**
and **+8.51%**, small **-1.57%** and **+8.90%**, and medium **+0.70%** and
**+0.02%** over two cycles.  `declined_after_largest` is retained as a fact
rather than a pass condition, because Windows heap trimming is nondeterministic
even when the runtime has released the input and activation vectors.

### 2026-08-12: default AVX-512 wide-context Conv tile

The AVX-512 eight-output tile for the PP-OCRv6 medium detector's `5x5` and
`7x7` context layers is now enabled by default for its narrow, runtime-checked
shape (`C_in >= 32`, `C_out >= 8`, and both spatial dimensions at least 32).
It reuses each input vector across eight output reductions; smaller shapes,
AVX2, NEON, and scalar targets retain their existing paths. Set
`PPOCR_DISABLE_AVX512_CONV_WIDE8=1` for a deployment-local A/B test.

On the local AVX-512 host with 12 workers, three warm-ups and twelve `en.ppm`
runs, medium CPU measured **424.234 ms** with the tile and **659.937 ms** with
`PPOCR_DISABLE_AVX512_CONV_WIDE8=1` (same model/input; shared-host variation
is reflected by the recorded medians **404.622 / 668.809 ms**).  Fresh
ONNX-Runtime decoded-text comparison remained **12/12** across tiny, small,
and medium on `16x16`, English, low-contrast, and `3000x80` inputs. The medium
two-cycle `64x64 閳?2560x1440 閳?64x64` ladder also passed all bounded-memory
checks, ending at **+0.19%** and **+2.72%** from its warmed baseline. Vulkan
primitive smoke remained numerically passing; hybrid selected only its
measured no-slower affine+Swish and depthwise+Swish samples in that run.

### 2026-08-12: medium worker and transpose-tile recheck

A fresh worker sweep after the wide-context default confirms the library's
automatic **16-worker** ceiling remains appropriate for the broad medium
graph, while tiny/small retain their shorter-kernel behavior: on the same
`en.ppm` setup, medium measured **191.664 / 205.187 / 185.906 ms** with
8/12/16 workers; tiny measured **28.434 / 28.797 ms** and small
**57.390 / 59.147 ms** at 12/16 workers.  The 16-worker medium CPU retest
(3 warm-ups, 20 runs) averaged **185.325 ms**; paired ONNX Runtime CPU on the
same image/run count averaged **260.194 ms**, a host-specific **28.8%** lower
native latency.

The AVX-512 four-output `ConvTranspose2x2` FPN experiment was rechecked too.
It measured **186.232 ms** versus **181.971 ms** for the established
single-output implementation (3 warm-ups, 10 runs), so it remains opt-in via
`PPOCR_ENABLE_AVX512_TRANSPOSE2X2_TILE4=1`; it is not promoted merely because
it reduces some input loads. This preserves the faster default and avoids a
regression on the current target.

### 2026-08-12: device-resident depthwise/pointwise Vulkan segment

The Vulkan backend now has a numerically checked two-dispatch MobileNet
`depthwise Conv -> 1x1 pointwise Conv` primitive.  It uploads the chain input
once, leaves the depthwise feature map in a mapped Vulkan storage buffer,
inserts an explicit compute shader write-to-read memory barrier, and reads
back only the pointwise result.  Its hybrid admission measures that complete
boundary against the corresponding CPU `DepthwiseConvBatch` plus
`PointwiseConvBatch` pair and rejects Vulkan when it is slower.

On the local Radeon 780M / AVX-512 host, the odd-shape batched numerical smoke
passed, but the representative `N=4, C=32, M=40, 31x29, 3x3` full-boundary
probe measured **0.836 ms Vulkan vs 0.271 ms CPU**.  Hybrid therefore keeps
the CPU path for this segment.  The primitive is deliberately not folded into
the production ONNX graph yet: its current model-level floating-point drift
needs a decoded-text tolerance gate before any performance claim or default
dispatch change.  This preserves exact production behavior while retaining a
real, barrier-validated GPU building block for subsequent fully-resident
segments.

The batch smoke executable now also prints its individual timed samples, so a
throughput claim can be audited for scheduler outliers rather than relying on
one aggregate number.  With four equal `en.ppm` pages, batch size four, 16 CPU
kernel workers, two warm-ups and eight timed runs, CPU serial閳妼atch results
were **120.826閳?5.704 ms (1.410x)** for tiny, **281.621閳?27.853 ms (1.236x)**
for small, and **859.114閳?44.174 ms (1.018x)** for medium. Hybrid under the
same batch settings measured **166.786閳?08.215 ms (1.541x)** tiny,
**331.511閳?51.555 ms (1.318x)** small, and **824.029閳?49.212 ms (1.100x)**
medium. Serial/batch OCR outputs were byte-for-byte equivalent in each run.
The hybrid figures are page-batching capacity gains; they do not claim that
the partial Vulkan graph is faster than CPU for every single page.

A fresh 12-cycle run after this lifetime change used a 23.488 MiB baseline and
finished at 26.664 MiB (+3.176 MiB); its cycle deltas ranged from -0.543 MiB to
+4.055 MiB with no monotonic growth. Windows heap retention makes the final
private-byte delta workload- and allocator-dependent, so the important release
criterion is the non-growing mixed-shape sequence, not a single final number.

After the batched constant-weight MatMul change, a medium-model two-cycle
mixed-size check covering `16x16`, `3000x80`, `80x3000`, and the 119-box
`1920x1080` UI page used **293,961,728 bytes** after warm-up; the two cycle
measurements were **811,008 bytes below** and **471,040 bytes below** that
baseline. This verifies that flattening a batch is a scheduling change, not a
per-shape tensor cache or an activation-retention increase. As above, it is a
Windows allocator-capacity observation rather than a universal leak proof.

After adding the Vulkan channel-affine batch buffers, the same medium mixed
sequence was rerun for two cycles with the default hybrid policy. Its
post-warm-up baseline was **290,508,800 bytes**; cycles finished at
**+5,214,208** and **+2,506,752 bytes**. The high-water movement is bounded
Windows allocator retention after the large UI page, not a per-cycle growth
trend; GPU affine admission was rejected on this host, so the extra persistent
Vulkan coefficient storage was not exercised by the CPU execution path.

After the BatchNorm-Swish in-place activation update, a fresh CPU medium
mixed-size soak (`16x16`, `3000x80`, `80x3000`, and the dense `1920x1080` UI
page) used **286,793,728 bytes** post-warm-up. Its next two cycles were
**9,101,312** and **5,144,576 bytes below** baseline. This is a bounded
allocator observation on Windows, but it confirms the source-buffer reuse does
not retain changing-shape recognition activations.

The captured C++ and ONNX Runtime reports are
`build/memory_soak_cpp_12.json` and `build/memory_soak_ort_12.json`. A
post-optimization 6-cycle C++ check is also retained as
`build/memory_soak_cpp_parallel4_6.json`: it finished 1.034 MiB below its
24.629 MiB post-warmup baseline, demonstrating that bounded concurrent
recognition does not retain every dynamic input shape.

### 2026-08-12: cross-page recognition batching

`OCR::RecognizeBatch` now keeps dynamic detector resizing per page, then
coalesces equal-width crops from all submitted pages into bounded NCHW
recognizer batches. This is useful for both CPU SIMD and the Vulkan batch-Y
dimension; it does not pad unrelated detector pages, does not change result
order, and releases each detector probability map before recognition starts.

On this Radeon 780M / Windows host, medium PP-OCRv6, four copies of `en.ppm`,
`rec_batch_size=8`, one recognizer worker and one detector worker:

| Path | Serial pages | Cross-page batched | Speedup |
|---|---:|---:|---:|
| CPU | 773.012 ms | 744.098 ms | 1.039x |
| Hybrid | 854.746 ms | 1249.711 ms | 0.684x |

The batch smoke test confirms exact public `Result` equivalence with serial
inference. Hybrid is intentionally not presented as a win here: the current
UMA Vulkan implementation admits each segment only when its complete transfer
+ dispatch + readback path is no slower than CPU, while the coalesced workload
still has significant CPU-only graph work. Four-image mixed-size CPU coverage
(`en`, low-contrast, wide strip, dense 1920x1080 UI; 128 results) also passed
the serial-result check at 1.004x. A tiny-model four-page CPU check measured
143.472 ms serial and 139.913 ms batched (1.025x). Re-run these workloads on
the deployment hardware; they are measurements, not portability claims.

### 2026-08-12: depthwise Vulkan batch kernel and hybrid batch gate

The Vulkan depthwise-convolution path now maps one workgroup to one NCHW
channel and up to 1,024 output pixels. Its `[KH,KW]` filter is cooperatively
staged in workgroup memory (up to `16x16`), while Vulkan's Y dimension remains
the independent crop-batch dimension. PP-OCRv6's MobileNet-style depthwise
filters therefore avoid reloading the same channel filter for every output
pixel. Larger exported filters safely use the existing global-memory fallback.

The Radeon 780M numerical smoke test passed for a padded `N=2, C=5, 7x9,
3x3` depthwise batch. Its latest complete synchronous measurement was
**0.44340 ms GPU / 0.35062 ms CPU**, so the strict H2D + dispatch + D2H
admission policy correctly retained the CPU SIMD path for that shape. This is
an optimized, batch-capable GPU primitive, not a claim that the incomplete
GPU-resident OCR graph is faster on this UMA GPU.

The same build also ran the public batch-equivalence harness on tiny PP-OCRv6
with `rec_batch_size=8`, one detector worker and one recognizer worker across
four varied pages (`en`, low-contrast, `3000x80`, dense `1920x1080`). All
**129** decoded results matched serial `Recognize()` exactly. The one-shot
local measurement was **1081.928 ms serial / 1167.667 ms batched** for CPU and
**1105.010 ms serial / 1075.921 ms batched** for hybrid. These scheduler
samples are intentionally retained as measurements rather than a universal
throughput promise; the dense mixed-width workload has limited equal-width
crop coalescing. A separate CPU-versus-hybrid dense-page result comparison was
also byte-for-byte identical.

### 2026-08-12: cross-crop CPU pointwise scheduling

The CPU 1x1-convolution path now accepts a real NCHW batch in one scheduling
call. On x86 it flattens `(crop N, output-channel tile)` into the persistent
AVX-512/AVX2 worker queue, retaining the existing per-channel FP32 reduction
order and ISA-specific four/eight-output vector kernels. ARM/NEON and scalar
builds retain their portable per-crop path. This removes repeated worker-pool
barriers for batchable recognizer projections without changing the model graph
or requiring ONNX Runtime.

The strict CPU-versus-ONNX Runtime decoded-text gate passed **4/4** varied
inputs (`en`, `16x16`, low-contrast, `3000x80`) for **tiny, small, and
medium** after this change. The five-page public batch gate (also adding dense
`1920x1080`) passed serial-result equivalence for all three models: tiny
**130**, small **126**, medium **129** results. A local one-worker CPU batch
sample measured **0.948x / 1.162x / 1.014x** page-batch throughput for
tiny/small/medium respectively; the mixed dimensions intentionally make this
an evidence point rather than a universal latency claim. Hybrid revalidation
also preserved exact serial results at **1.003x / 1.036x / 0.983x**.

On `en.ppm`, 2 warm-ups and 6 timed CPU runs with 16 workers, the current
native means were **44.193 ms / 84.184 ms / 256.530 ms** for tiny/small/medium.
The matched ONNX Runtime CPU samples were **127.640 ms / 244.623 ms /
513.763 ms**, corresponding to **2.89x / 2.91x / 2.00x** native latency
advantages on this host. Windows scheduling remains variable; these are
repeatable local observations, not a portable capacity guarantee.

The five-size, two-cycle memory soak (`16x16`, `en`, low-contrast, `3000x80`,
`1920x1080`) remained bounded. Its warmed baselines and final deltas were:
tiny **113,876,992 / +3,874,816 bytes**, small **150,462,464 / -2,027,520
bytes**, and medium **385,916,928 / -3,629,056 bytes**. The small movements are
Windows allocator high-water behavior rather than a leak assertion.

### 2026-08-12: batched residual 1x1 projection scheduling

Fused `Conv(1x1) -> Add`, `Conv(1x1) -> Add -> ReLU`, and `Conv(1x1) -> Add
-> Swish` now have NCHW batch kernels. On x86, one work queue covers all
`(crop, output-channel SIMD tile)` tasks: AVX-512 uses the established
four/eight-output tiles and AVX2 uses four-output tiles. ARM/NEON and scalar
builds safely retain the per-crop implementation. The Swish variant completes
the batched fused convolution/add first, then applies the existing vector
Swish once across the whole contiguous NCHW allocation; FP32 reduction and
activation order are unchanged.

`ppocr_kernel_smoke` verifies the new Add, Add+ReLU, and Add+Swish batch
kernels against their single-crop counterparts for both SIMD-tail and large
work cases. It also passed with `PPOCR_DISABLE_AVX512=1`, exercising the AVX2
fallback. `PPOCR_DISABLE_RESIDUAL_POINTWISE_BATCH=1` is available solely as a
per-process A/B escape hatch; it restores the prior per-crop calls.

The four-image decoded-text comparison with ONNX Runtime remained **4/4** for
each tiny, small, and medium model. The five-size public batch-equivalence
gate also matched all serial `Result`s exactly: tiny **130**, small **126**,
medium **129**. One local CPU sample (one detector/recognizer worker,
`rec_batch_size=8`) measured serial/batch times of **909.483/907.159 ms**
(tiny, **1.003x**), **2780.569/2882.623 ms** (small, **0.965x**), and
**6781.126/6750.499 ms** (medium, **1.005x**). Mixed crop widths limit
coalescing, so these are recorded measurements rather than a portability
claim. Hybrid correctness was retained on the same corpus: tiny **1.134 s /
1.132 s** (0.994x), small **3.978 s / 3.515 s** (1.132x), and medium **7.024 s
/ 7.464 s** (0.941x). The hybrid backend still admits each Vulkan segment only
after a complete transfer + dispatch + readback no-slower check; the project
does not claim a fully GPU-resident graph or enable `gpu_only`.

### 2026-08-12: cross-crop CPU depthwise scheduling

The MobileNet depthwise CPU fallback now accepts the leading NCHW batch
directly. It flattens independent `(crop, channel)` jobs into one persistent
worker-pool pass, while each job continues to use the existing runtime-selected
AVX-512, AVX2, NEON, or scalar depthwise kernel. This removes repeated batch
loop scheduling around the dominant depthwise backbone stages without changing
the per-channel FP32 multiply/add order. The Vulkan depthwise route remains
separate and still receives the entire NCHW batch in one dispatch whenever its
complete transfer + dispatch + readback admission measurement is no slower
than CPU.

The extended kernel smoke tests padded/stride-1 `3x3` and padded/stride-2
`5x5` depthwise batches against the established per-image implementation; both
the normal AVX-512 run and `PPOCR_DISABLE_AVX512=1` AVX2 fallback passed. The
four-image decoded-text comparison with ONNX Runtime remained **4/4** for each
tiny, small, and medium model. The five-size batch-result gate likewise
remained exact (tiny **130**, small **126**, medium **129** results).

On the varied five-page corpus with `rec_batch_size=8` and one recognition
worker, this local CPU sample measured serial/batch throughput of **1.096x**
(tiny: 1000.988/913.176 ms), **0.992x** (small: 2855.215/2878.902 ms), and
**1.006x** (medium: 6922.651/6881.270 ms). Crop-width diversity constrains
batch density, so the figures are intentionally recorded as host-specific
measurements. Hybrid preserved serial-result identity at **1.011x**, **1.058x**
and **0.924x** for tiny/small/medium; GPU segments were retained only when the
strict no-slower admission test passed. A renewed five-size/two-cycle memory
soak stayed bounded: tiny baseline **99,737,600**, final **+716,800** bytes;
small baseline **148,520,960**, final **+1,306,624** bytes; medium baseline
**384,311,296**, final **-3,907,584** bytes.

### 2026-08-12: batched ordinary 3x3 convolution scheduler

The group-1 non-pointwise convolution path now supports NCHW batches. For the
common `3x3` stride-1 and stride-2 OCR shapes it flattens crop and established
AVX output-channel tiles into one persistent-pool submission: AVX-512 retains
its four/eight-output tile selection; AVX2 retains four-output stride-1 tiles
and its stride-2 kernel. ARM/NEON and non-3x3 geometries continue through the
existing per-image `Conv2d` path. This changes scheduling only, not the output
channel reduction order. `PPOCR_DISABLE_CONV_BATCH=1` restores the previous
per-crop dispatch for deployment A/B work.

The extended kernel smoke checks stride-1 tail, large AVX-512 tile, and
stride-2 ReLU batches against the prior single-image path; both AVX-512 and
forced AVX2 runs pass. ONNX Runtime decoded-text comparisons again passed
**4/4** for each tiny/small/medium model. On the five-size corpus with one
recognizer worker and `rec_batch_size=8`, the latest CPU serial/batch samples
were tiny **839.965/822.967 ms** (1.021x), small **2602.329/2656.020 ms**
(0.980x), and medium **6507.553/6500.383 ms** (1.001x), all with exact public
batch results. A consecutive A/B sample with `PPOCR_DISABLE_CONV_BATCH=1`
measured **1.023x / 0.966x / 1.005x** on the same mixed-width suite; host noise
and limited same-width coalescing outweigh this small scheduling change there,
so no universal speedup is claimed. Hybrid result equivalence held at
**1.009x / 0.778x / 0.979x**; Vulkan continues to be selected only by its
complete no-slower segment measurement.

### 2026-08-12: batched pointwise HardSwish projection

The remaining fused `1x1 Conv -> HardSwish` CPU path now shares the existing
cross-crop pointwise SIMD scheduler. It performs all NCHW projections through
`PointwiseConvBatch`, then runs one contiguous exact HardSwish pass over the
final allocation, instead of issuing the projection/activation once per crop.
This preserves the Conv FP32 reduction order and the canonical HardSwish
formula; AVX-512, AVX2, NEON, and scalar dispatch remain runtime portable.

Kernel smoke now compares both tail and large batched HardSwish projections to
the established per-crop routine. Normal AVX-512 and forced AVX2 execution
passed. The four-input ONNX Runtime decoded-text gate again passed **4/4** for
tiny, small, and medium, and the varied five-page batch gate remained exactly
identical to serial results (130 / 126 / 129 results). Latest CPU
serial/batch samples on that suite were tiny **901.083/842.421 ms** (1.070x),
small **2515.113/2599.535 ms** (0.968x), and medium
**6177.222/6239.080 ms** (0.990x). Hybrid samples were **0.996x / 1.250x /
0.972x**. These mixed-width measurements are evidence, not portability
claims; each Vulkan segment continues to require its own complete no-slower
transfer/dispatch/readback admission.

### 2026-08-12: batched FPN 2x2 transposed convolution

The detector's `2x2`, stride-two `ConvTranspose` FPN fast path now accepts
NCHW batches. It flattens `(crop, output-channel tile)` into one scheduling
pass: AVX-512 keeps its optional four-output reuse tile (or one-output default)
and AVX2 uses its established channel kernel; ARM/scalar builds retain the
same portable per-image routine. This preserves each output channel's ordered
input-channel accumulation and eliminates repeated batch-loop scheduling.

`ppocr_kernel_smoke` now validates small tail and large square batch cases
against the prior per-image FPN kernel, on both normal AVX-512 and forced AVX2
paths. The tiny/small/medium ONNX Runtime decoded-text checks again passed
**4/4** each. The five-size public batch gate stayed serial-identical at
130/126/129 results. Latest CPU serial/batch observations were tiny
**808.237/821.381 ms** (0.984x), small **2467.217/2557.340 ms** (0.965x), and
medium **6063.649/6071.106 ms** (0.999x). This page mix does not batch detector
shapes and has diverse recognition widths, so it should not be read as a
universal throughput claim. Hybrid exact-result samples were **1.019x / 0.859x
/ 0.974x**. The renewed medium five-size/two-cycle soak remained bounded
(baseline **384,970,752** bytes; final delta **+3,231,744** bytes), and Vulkan
smoke retained strict complete-segment admission.

### 2026-08-12: batched ordinary-convolution Vulkan path

Vulkan hybrid execution now also covers ungrouped, non-dilated ordinary NCHW
convolution (`3x3` and `2x2` detector forms), in addition to the existing
pointwise and depthwise paths.  The new shader maps a real crop batch to its
Y dimension and writes four adjacent output values per invocation where
possible.  Learned `[M,C,KH,KW]` weights and `[M]` bias retain their compact
layouts and are cached for immutable ONNX initializers; an immediately fused
ReLU can complete before the only D2H copy.  Asymmetric-stride, grouped, and
dilated convolutions deliberately retain the established CPU SIMD path.

This remains a *hybrid* primitive, not a GPU-only claim.  Every shape is
first validated against the CPU batched convolution and admitted only when its
complete H2D + dispatch + fence wait + D2H time is no slower.  The new Vulkan
smoke test covers padded stride-one and stride-two `N=3,C=4,M=7,3x3` batches,
including fused ReLU and scalar output tails.  On the local AMD Radeon 780M,
the larger admission sample (`N=4,C=24,M=40,31x29,3x3,ReLU`) measured
**3.53696 ms GPU / 0.83228 ms CPU** after the shared-filter tile optimization,
so the production policy correctly kept
that shape on CPU rather than adding GPU overhead.

After the change, the five-size public batch-equivalence harness preserved
serial output identity for all official model sizes under `PPOCR_BACKEND=hybrid`
with recognizer batch size 8: tiny **130** results (**1188.492 / 1163.981 ms**, 
**1.021x** serial/batch), small **126** (**3425.233 / 3507.859 ms**, **0.976x**),
and medium **129** (**7949.857 / 8198.539 ms**, **0.970x**).  These are
host-specific mixed-size scheduler samples, not a universal GPU-performance
promise.  `gpu_only` remains unavailable until the entire detector and
recognizer graph is device-resident.

### 2026-08-12: final-use BatchNormalization storage reuse

Standard inference `BatchNormalization` nodes now use the same NCHW
channel-affine kernel as fused normalization paths.  When graph liveness
proves that the input has reached its final use, the executor applies the
precomputed `scale / sqrt(variance + epsilon)` and `bias - mean * scale`
directly in that dying allocation, then renames it to the output.  This avoids
one complete activation allocation and copy for each eligible unfused node.
The official model bundle folds most normalisation nodes at load time, so this
extends compatible PP-OCRv6 graph coverage rather than being represented as a
separate current-model speedup.  The new
`BatchNormAffine` primitive is alias-safe and dynamically dispatches through
AVX-512, AVX2, NEON, or scalar `ScaleShift`; it is used in both `Run()` and
the recognizer's real `RunCtcTop1()` path.  `PPOCR_DISABLE_BATCHNORM_INPLACE=1`
is available for deployment A/B checks.

Kernel coverage now includes separate out-of-place and in-place batch-affine
numeric tests, and passed with both default AVX-512 dispatch and forced AVX2.
The post-change C++/ONNX Runtime decoded-text gate passed **4/4** varied
images (`16x16`, normal English, `3000x80` strip, noisy low-contrast) for
both tiny and medium.  A five-size medium two-cycle memory soak remained
bounded at a **457,703,424-byte** post-warm-up baseline; cycles ended at
**-9,195,520** and **+1,744,896 bytes**, consistent with allocator
high-water reuse rather than per-cycle growth.  The same five-size hybrid
tiny batch equivalence test returned all **130** decoded results identically
to serial inference (**1123.880 / 1118.024 ms**, **1.005x** serial/batch).
These are validation measurements, not a claim that every normalisation node
or GPU segment is faster on every host.

### 2026-08-12: batched Vulkan FPN transpose-convolution path

The detector FPN's `ConvTranspose(2x2, stride=2)` projection now has a
dedicated batched Vulkan primitive.  Its no-padding geometry has no overlap
between output tiles, so each output pixel maps directly to one input pixel
and one `[Cin,Cout,2,2]` filter tap; this avoids an expanded intermediate.
The kernel accepts NCHW batches, caches immutable model parameters in the
persistent Vulkan context, and uses one output readback.  The command
dispatcher was also corrected to submit workgroups (256 four-float
invocations per group), rather than accidentally submitting that many times
the required groups.

The hybrid executor now recognizes this exact ONNX form and uses it only
after an end-to-end CPU comparison for the exact batch/shape.  The comparison
includes H2D, dispatch, fence wait, D2H, and FP32 output validation; rejected
shapes continue through the AVX-512/AVX2/NEON/scalar CPU kernel with no output
change.  Vulkan smoke now validates an odd-size `N=3,Cin=5,Cout=7,11x13`
batch against scalar FP32 reference values, including its vector tail.

On the local AMD Radeon 780M, the FPN admission sample
(`N=4,Cin=32,Cout=40,31x29`) measured **2.48820 ms GPU / 0.60534 ms CPU**,
so it was correctly rejected rather than being presented as an OCR speedup.
The post-change five-size tiny hybrid batch-equivalence run remained exactly
serial-identical with **130** decoded results and measured
**1098.189 / 1079.664 ms** serial/batch (**1.017x**).  This is a real
mixed-size batch validation on this host, not a general GPU throughput claim.
`gpu_only` remains unavailable until the full detector and recognizer graphs
can stay device-resident.

### 2026-08-12: FPN transpose shared-filter GPU tile and multi-model batch gate

The Vulkan FPN transpose primitive now selects a shared-filter workgroup form
when its `[Cin,2,2]` row fits the 4,096-float local-memory budget.  A
workgroup stages the non-contiguous ONNX `[Cin,Cout,2,2]` row into contiguous
shared memory once, then computes a 1,024-pixel tile for one output channel.
The staging explicitly preserves the original increasing-input-channel FP32
reduction order.  Wider exported layers retain the generic batch kernel.  The
odd `N=3,Cin=5,Cout=7,11x13` Vulkan numerical test continues to compare every
result with scalar reference values.

On the local Radeon 780M, the complete-transfer FPN admission sample improved
from the preceding **2.48820 ms** to **2.15242 ms GPU**; CPU measured
**0.45748 ms**, so hybrid still correctly selects CPU for that isolated node.
This is intentional: hybrid uses Vulkan when the measured complete segment is
equal or faster, rather than increasing CPU offload at the cost of latency.

The five-size public hybrid batch gate (`16x16`, normal English,
low-contrast, `3000x80`, and dense `1920x1080`; recognizer batch 8, one
recognizer worker) remained serial-identical for all official models: small
**126** results (**2751.279 / 3384.936 ms**, **0.813x**) and medium **129**
results (**7358.610 / 7808.435 ms**, **0.942x**) in this shared-host sample;
tiny's same gate was **130** results (**1098.189 / 1079.664 ms**, **1.017x**).
The C++/ONNX Runtime medium decoded-text gate also passed **4/4** distinct
sizes/aspect ratios (`16x16`, English, low-contrast, wide strip).  The
repository's paired benchmark scripts now accept `-Model tiny|small|medium`,
and `bench/compare_models.ps1` runs all three across the same five-size corpus
for deployment-specific ORT comparison.  Mixed-size batching is not asserted
as a universal speedup; output equivalence and bounded work are the gate.

For a fresh paired single-page sanity sample after this GPU-tile change
(`en.ppm`, two warm-ups, five runs, 16-thread CPU budget, CPUExecutionProvider
for ONNX Runtime), all three official sizes remained faster than ORT:

| Model | C++ hybrid mean | ORT CPU mean | C++ improvement |
|---|---:|---:|---:|
| tiny | 95.490 ms | 128.475 ms | 25.67% |
| small | 157.035 ms | 270.218 ms | 41.89% |
| medium | 263.851 ms | 478.374 ms | 44.84% |

The host is shared and short-run tiny measurements have greater variance; the
table is current measured evidence, not a portable guarantee.  It uses the
same complete detector+recognizer OCR work in both implementations.

### 2026-08-12: terminal CTC guard and scalar-reference coverage

`RunCtcTop1()` already avoids materializing the terminal
`[N,T,6906]` Softmax activation: it keeps final logits, computes only CTC
top-1 indices and probabilities for emitted characters, then releases the
logits at function exit.  This revision narrows the fast-path eligibility to
a verified class-axis terminal Softmax (`axis=-1` or `axis=2`) on the official
`[N,T,V]` exports; a terminal Softmax over any other axis now fails fast rather
than silently applying CTC semantics to the wrong dimension.  The low-level
kernel smoke now compares SIMD-dispatched CTC against a scalar reference for
both a tail-heavy `N=3,T=17,V=31` case and production-width
`N=4,T=41,V=6906`, including blank and repeated-token paths.

Validation after the guard: the expanded CTC kernel smoke passed under both
default AVX-512 dispatch and forced AVX2; C++/ONNX Runtime decoded text passed
**4/4** diverse inputs for tiny and small, while the preceding medium gate was
also **4/4**.  The five-size tiny hybrid batch gate remained serial-identical
at **130** results (**1207.436 / 1297.085 ms**, **0.931x** on this shared
host). A medium five-size/two-cycle memory soak stayed bounded at a
**449,376,256-byte** post-warm-up baseline; cycles ended at **+7,770,112** and
**+5,087,232 bytes**, which is allocator high-water reuse rather than monotonic
per-cycle growth.  Radeon 780M Vulkan smoke remained numerically passing; its
full-transfer policy continued to select GPU only for shapes that measured no
slower than CPU.

### 2026-08-12: hybrid detector batch admission and benchmark controls

`RecognizeBatch()` now permits its existing exact-shape detector NCHW batches
when `Backend::hybrid` is selected as well as on CPU. This does not change
resize semantics or pad differently sized pages: pages are still grouped only
when their independently calculated detector width and height match exactly.
Every Vulkan operator continues to make a separate batch-shape admission
decision that includes H2D, dispatch, fence wait, D2H, and numerical
validation. A rejected operator uses its CPU SIMD batch kernel, so enabling a
larger detector batch never forces a slower partial-GPU path.

`ppocr_bench` now reads `PPOCR_DET_BATCH_SIZE`; `ppocr_batch_smoke` also
reports both effective detector and recognizer batch sizes (and accepts
`PPOCR_REC_WIDTH_BUCKET`). This makes the public batch path reproducible for
deployment A/B work.

On the local Radeon 780M system, eight equal `en.ppm` pages, one page worker,
`rec_batch_size=8`, and `det_batch_size=8` remained exactly serial-equivalent.
One fresh steady-state sample measured CPU **296.587 / 181.402 ms**
(**1.635x**) and hybrid **343.533 / 219.900 ms** (**1.562x**) for serial versus
batched execution. Hybrid with `det_batch_size=1` measured **274.675 /
232.065 ms** (**1.184x**) in a separate run, so this host benefited from
allowing the same-shape detector batch. These are host-specific measurements,
not a claim that every GPU or page mix will have the same ratio.

Regression coverage after the change: normal and AVX2-forced
`ppocr_kernel_smoke` passed; the Vulkan primitive smoke passed on AMD Radeon
780M; and decoded text matched ONNX Runtime for all four varied images in each
of tiny, small, and medium (**4/4 per model**). `gpu_only` remains explicitly
unavailable until the full detector and recognizer graph is device-resident.

### 2026-08-12: fused Vulkan FPN resize/add batch primitive

The FPN's graph-proven `Resize(nearest, 2x) -> Add(lateral)` sequence now has
a Vulkan `NCHW` batch primitive. It uploads the small source map and lateral
residual once, generates each nearest-neighbour value and sums it in one
shader dispatch, then reads only the completed result. The short-lived
expanded resize tensor is therefore never materialized on either CPU or GPU.
Odd dimensions and four-page batch addressing are covered by the Vulkan
smoke test.

Hybrid admission is shape-specific and strict: the measured path includes
source/residual H2D copies, shader dispatch, fence wait, D2H copy, and every
FP32 output comparison against `NearestResize2xAdd`. On this AMD Radeon 780M,
the representative `N=4,C=24,31x29` probe measured **0.47230 ms GPU** versus
**0.06686 ms CPU**, so it is correctly retained on AVX-512 CPU rather than
being forced onto the GPU. This primitive is nevertheless batch-ready for
hardware where the complete path wins.

Post-change validation: default and forced-AVX2 kernel smoke passed; Vulkan
smoke passed; tiny CPU and hybrid decoded-text comparison against ONNX Runtime
passed **4/4** varied inputs; and CPU tiny five-size/two-cycle memory soak
finished at **-139,264** and **-13,721,600 bytes** relative to a
**146,825,216-byte** post-warm-up baseline. A fresh eight-identical-page,
batch-8 smoke sample was exact-equivalent to serial: CPU **303.220 / 181.519
ms (1.670x)** and hybrid **336.976 / 213.733 ms (1.577x)**, serial/batched.
For a single `en.ppm` page with batch sizes one, CPU averaged **37.206 ms**;
hybrid averaged **37.649 ms**, so this shared host's current partial offload
does not claim a whole-pipeline GPU gain. `gpu_only` remains unavailable until
the entire detector and recognizer graphs are device-resident.

### 2026-08-12: recognizer CTC-path lifetime and Vulkan GEMM coverage

The production `RunCtcTop1()` recognizer executor now directly runs fused
`1x1 Conv + shortcut` nodes, matching the ordinary executor rather than
falling through its generic node path. This removes graph-input-vector setup
on the dense residual path and releases the source feature map immediately
after the fused kernel writes its destination. It is shared by tiny, small,
and medium; the three-model CPU/hybrid decoded-text regression against ONNX
Runtime remained **4/4 per model** over tiny, normal English, noisy
low-contrast, and ultra-wide input dimensions.

Vulkan additionally now implements row-major projection GEMM (`[R,K] x
[K,C] + bias`) for recognizer attention and CTC-head-compatible layouts. It
has a numerical smoke for odd `37x29x103` dimensions and a strict full-boundary
admission probe. The Radeon 780M measured the representative `64x128x512`
case at **0.67616 ms GPU** versus **0.15548 ms CPU**, so hybrid correctly keeps
this shape on CPU. The same smoke measured GPU wins for selected batch
depthwise and affine+Swish shapes, which hybrid admits automatically. This
keeps GPU work data-driven and prevents an isolated GPU primitive from
silently increasing complete-OCR latency.

The AVX-512 FPN resize/add kernel was also tightened to load one contiguous
16-float source vector per iteration before two indexed broadcasts; default
and forced-AVX2 kernel smoke pass. On a fresh 16-worker `en.ppm` run (three
warmups, ten timed runs), CPU means were **33.535 / 68.351 / 215.205 ms** and
hybrid means **32.155 / 76.156 / 258.877 ms** for tiny/small/medium. Current
paired ONNX Runtime means from the same session were **176.288 / 229.349 /
572.251 ms**, so CPU remains the deployment recommendation for small/medium
on this UMA GPU while retaining a large native-vs-ORT advantage.

A five-size tiny batch gate (`16x16`, English, low contrast, `3000x80`, dense
`1920x1080`; batch 8) returned all **130** results identically. This mixed
corpus measured **874.808 / 987.314 ms** CPU serial/batched and **1096.915 /
1107.025 ms** hybrid serial/batched, demonstrating that heterogeneous work is
not mislabeled as a throughput win. Its two-cycle memory soak stayed below its
**146,587,648-byte** post-warm-up baseline by **14,483,456** and
**14,401,536 bytes**. `gpu_only` remains explicit-unavailable until complete
detector and recognizer device residency is implemented and verified.

### 2026-08-12: CTC residual GPU admission and true page-batch validation

`RunCtcTop1()` is the production recognizer path, so its fused `1x1 Conv +
shortcut` blocks now use the same batched Vulkan admission path as the generic
executor.  This closes a policy gap where recognizer residual blocks could not
be considered for hybrid execution even when their exact NCHW batch shape had
already been measured.  The decision remains conservative and shape-local:
Vulkan is used only after the full input/residual upload, dispatch, fence wait,
readback, and CPU-SIMD numerical check are no slower than the CPU block.

The change was verified on AMD Radeon 780M with kernel SIMD smoke, Vulkan
primitive smoke, and identical decoded CPU OCR output. A four-page equal-size
tiny batch (`4 x en.ppm`, `det_batch=4`, `rec_batch=4`, one page-scheduler
worker) returned all eight text lines identically and measured:

| Backend | Serial pages | True NCHW page/crop batch | Speedup |
|---|---:|---:|---:|
| CPU | 148.554 ms | 91.244 ms | 1.628x |
| Hybrid | 138.904 ms | 94.528 ms | 1.469x |

This is a same-shape batching result, not a claim for arbitrarily mixed page
sizes. The existing scheduler still groups detector inputs only when their
native PP-OCR resize shape matches and buckets recognizer crops by padded
width, avoiding semantic changes from artificial image padding.

Fresh `en.ppm` end-to-end measurements (16 CPU workers, 3 warmups, 10 timed
runs) show the native engine remains faster than the paired CPU ONNX Runtime
baseline. These are host-specific AMD Radeon 780M / AVX-512-host measurements,
not portable guarantees:

| Model | C++ CPU | C++ hybrid | ORT CPU | CPU latency reduction vs ORT |
|---|---:|---:|---:|---:|
| tiny | 38.215 ms | 37.812 ms | 127.162 ms | 69.9% |
| small | 78.114 ms | 76.542 ms | 220.022 ms | 64.5% |
| medium | 220.072 ms | 240.608 ms | 487.076 ms | 54.8% |

The medium hybrid result is intentionally reported as slower than CPU. Its
individual Vulkan segments may still be admitted only where their complete
boundary beats SIMD; varying transfer/cache pressure on this shared UMA host
can make end-to-end hybrid slower. `gpu_only` still fails explicitly because
the entire PP-OCRv6 graph is not yet Vulkan-resident.

### 2026-08-12: default CPU worker budget tuned for PP-OCR batches

The persistent SIMD worker pool now defaults to **12** workers (still
overrideable with `PPOCR_BENCH_THREADS`). This is a measured scheduling/cache
decision, not a hardware-core-count assumption: on the local AVX-512 host,
the default PP-OCR graph's short detector/recognizer channel tiles saturated
useful memory bandwidth before 16 workers and the extra threads made all three
model sizes slower. The public crop scheduler default is aligned to the same
12-worker budget, avoiding nested page-level oversubscription.

On `en.ppm` with CPU-only execution, three warmups and eight timed runs, the
thread sweep measured the following mean end-to-end latency:

| Model | 8 workers | 12 workers (new default) | 16 workers |
|---|---:|---:|---:|
| tiny | 30.140 ms | 32.457 ms | 35.257 ms |
| small | 70.752 ms | 62.472 ms | 71.507 ms |
| medium | 226.918 ms | 215.528 ms | 226.332 ms |

Tiny stays fastest at eight workers on this host, while the new default gives
the best balanced small/medium result and avoids the former 16-worker
regression. Deployments optimizing only tiny can use
`PPOCR_BENCH_THREADS=8`; larger-server CPUs can continue to tune the existing
override.

Post-change validation covered default and forced-AVX2 kernel smoke, Radeon
780M Vulkan smoke, all three `en.ppm` models, and true NCHW batches. The
five-size corpus (`16x16`, English, low contrast, `3000x80`, dense
`1920x1080`) decoded **130** tiny-model results identically in CPU and hybrid
batch scheduling. Its CPU serial/batched timing was **446.898 / 408.895 ms**;
hybrid was **805.625 / 440.226 ms**. The two-cycle tiny memory soak over that
same five-size corpus ended **10,407,936** and **11,051,008 bytes below** its
128,106,496-byte post-warm-up baseline.

### 2026-08-12: fused Vulkan recognizer GEMM + Swish

The Vulkan recognizer projection primitive now has a `GEMM + Swish` mode for
the graph-fused MLP projection. It accepts the flattened rows of a real crop
batch, caches immutable weights/biases as before, and applies the exact Swish
while writing the GPU result. This removes the former complete CPU-side Swish
traversal after a successful hybrid GEMM dispatch. The shader path is covered
by an odd-tail `41x29x103` numerical smoke case and an end-to-end admission
probe against CPU SIMD `GEMM + Swish`; hybrid still uses it only when the full
upload, dispatch, fence, and readback time is no slower. This is additional
batch-aware GPU coverage, not a claim that the complete PP-OCR graph is yet
device-resident or that `gpu_only` is available.

Validation after the change passed the normal and forced-AVX2 kernel smoke
suites, plus the Radeon 780M Vulkan suite (including the new numerical
GEMM+Swish tail case). On that smoke run the complete-boundary GEMM+Swish
probe measured **0.65962 ms GPU vs 0.15050 ms CPU**, so the dynamic policy
correctly retained CPU for that transfer-bound shape. Tiny-model decoded-text
comparison against ONNX Runtime again passed **4/4** diverse inputs. Four
identical `en.ppm` pages also remained serial-equivalent: CPU measured
**151.283 / 103.396 ms (1.463x)** serial/batched, hybrid
**133.678 / 109.307 ms (1.223x)**. These are local samples; the stricter
per-segment admission rule remains the GPU performance authority.

### 2026-08-12: terminal CTC projection memory experiment

The recognizer executor now contains an optional experimental
`PPOCR_ENABLE_FUSED_TERMINAL_CTC=1` path that computes a terminal
`MatMul(+bias) -> Softmax -> CTC Top-1` directly from one reusable vocabulary
row buffer. It eliminates the transient `[N,T,V]` logits allocation and
preserves the dense projection's K-order plus exact selected-class Softmax
probability. It is deliberately opt-in: on the local AVX-512 CPU the scalar
vocabulary-row implementation is not a stable latency win for every official
model, although it is useful for memory-constrained deployments and passed
the text regression on the exercised corpus. The default continues to use the
proven SIMD GEMM plus vectorized CTC scan until a native SIMD fused terminal
kernel demonstrates a three-model speedup.

The row-buffer path is now covered in the SIMD kernel smoke with both a
small odd-tail projection and a `4x41x6906` recognizer-sized projection.
The portable regression corpus was also expanded beyond the original four
shapes: blank `800x600`, English and CJK single lines, a mixed three-line
receipt, and two-line CJK all decode with ONNX Runtime text parity for tiny,
small, and medium (**15/15 per model**). Deliberately adversarial rotation and
extreme portrait-strip images are retained as diagnostic samples, but are not
used as decoded-text parity gates because the two independently implemented
detector postprocessors can select different unstable low-confidence boxes.

### 2026-08-12: cooperative Vulkan GEMM tile and larger page batches

The Vulkan recognizer GEMM now has a cooperative **8-row 鑴?128-column** FP32
tile for ordinary (non-Swish) projections. Each workgroup stages a 16-wide K
slice of A and B, reusing B across eight rows rather than rereading it per
output element. The kernel keeps K tiles in ascending order, retains the
existing scalar-tail path, and is selected only after the existing exact
CPU-versus-full-boundary admission check. `gpu_only` remains unavailable: this
is a batch-aware primitive within the hybrid executor, not a device-resident
PP-OCR graph.

The new path is covered by Vulkan numerical smoke with a tail-heavy
`13鑴?7鑴?73` GEMM (rows, K, columns), in addition to the existing odd-tail and
GEMM+Swish cases. Normal SIMD smoke and Vulkan smoke passed on the local
AVX-512 / Radeon 780M host. Hybrid decoded-text parity against ONNX Runtime
again passed 4/4 diverse inputs for both tiny and medium (`16x16`, English,
low-contrast, and `3000鑴?0`).

For an eight-page same-shape `en.ppm` batch with `det_batch_size=8`,
`rec_batch_size=8`, and one recognizer scheduler worker, the post-change
hybrid measurements were tiny **304.188 / 225.182 ms (1.351x)** and medium
**2029.114 / 2019.849 ms (1.005x)** serial/batched. A four-shape hybrid batch
gate (`16x16`, English, low-contrast, `3000鑴?0`; batch sizes 4) remained
serial-identical for all models: tiny **186.390 / 176.957 ms (1.053x)**, small
**379.836 / 371.670 ms (1.022x)**, medium **1209.497 / 1215.288 ms (0.995x)**.
These are one host's steady-state samples, not a universal GPU claim; the
per-shape GPU admission continues to reject transfer-bound work.

After the latest pass, default CPU/hybrid batch regression again produced
serial-identical results across four different page shapes (`16x16`, English,
low-contrast, and `3000x80`). With bounded `det_batch_size=4` and
`rec_batch_size=4`, the local CPU/hybrid batch timings were tiny
**176.722/185.384** and **170.369/165.231 ms**; small **343.559/301.114** and
**374.170/293.868 ms**; medium **1317.190/1291.101** and
**1163.054/1122.104 ms**. A medium two-cycle varied-size memory soak started
at **495,898,624 bytes** after warm-up and ended **9,752,576** then
**10,174,464 bytes below** that baseline. These are host-local measurements,
not a general ONNX Runtime comparison claim.

### 2026-08-12: optional PP-OCRv6 vector GELU validation

The existing optional recognizer vector tanh-GELU path
(`PPOCR_APPROX_GELU=1`) was revalidated on AVX-512/AVX2; NEON/scalar retains
its portable fallback. This removes scalar `erf` from the MLP activation hot
path. The exact-Erf ONNX path remains the default, and the model-specific
vector path was checked against ONNX Runtime decoded text rather than being
assumed numerically interchangeable.

With 12 worker threads, three warmups and ten `en.ppm` runs, the scalar-Erf
baseline measured **29.097 / 60.268 / 202.201 ms** for tiny/small/medium; the
optional vector path measured **23.577 / 53.054 / 191.974 ms** in the same
session (**19.0% / 12.0% / 5.1%** lower mean latency). A longer shared-host
30-run medium sample was noisier, so the 10-run paired sweep is recorded as
an optimization signal, not a portable latency guarantee.

The decoded-text parity gate passed **4/4** diverse dimensions against ONNX
Runtime for each official model size (tiny, small, medium): `16x16`, English,
low-contrast, and `3000x80` ultra-wide. CPU and hybrid also produced identical
Chinese tiny-model text for the CJK sample. The kernel smoke still passes with
normal AVX-512 dispatch and forced AVX2 fallback.

### 2026-08-12: AVX-512 batch-GEMM dispatch refinement

The AVX-512 dense-GEMM path now selects its four-row by 64-column micro-kernel
per projection shape instead of applying its extra register pressure to every
recognizer tier. It stays enabled only for `K >= 384` (and can be disabled for
A/B with `PPOCR_DISABLE_AVX512_GEMM4ROW=1`); AVX-512 閳?AVX2 閳?scalar/NEON
runtime dispatch remains unchanged for all x86 and Arm targets. This keeps
large medium-model projections eligible to reuse each immutable B vector over
four output rows, while tiny/small use the lighter proven path.

Fresh `en.ppm` CPU measurements (three warm-ups, ten runs, 12 kernel workers)
were tiny **29.699 ms**, small **59.577 ms**, and medium **196.789 ms**. The
previous all-shape four-row experiment measured small **82.637 ms** and medium
**267.150 ms** in the same benchmark configuration, so the new admission
guard is retained. A dense `1920x1080` medium page (122 boxes,
`rec_batch_size=4`, `rec_parallelism=4`, one warm-up, three runs) measured
**4,496.430 ms** with the wide tile enabled and **4,426.272 ms** with it
disabled; therefore the dense-page default now avoids that tile whenever its
projection depth is below the measured `K=384` cutoff. These are
host-local figures, not an ONNX Runtime claim.

The normal AVX-512 kernel smoke passed after the change, as did the complete
Radeon 780M Vulkan smoke suite. The latter again verified all GPU batch
primitives numerically and dynamically admitted only full-boundary winners:
affine+Swish **0.17342 vs 0.34950 ms**, depthwise **0.34814 vs 0.47170 ms**,
depthwise+Swish **0.35624 vs 0.60330 ms**, and depthwise+HardSwish
**0.34528 vs 0.41760 ms** (GPU vs CPU). Transfer-bound GEMM and other
primitives stayed on CPU; `gpu_only` is still explicitly unavailable until
the complete PP-OCR graph is device-resident.

### 2026-08-12: diverse-size regression and preprocessing arithmetic reduction

The preprocessing resize path now folds PP-OCRv6's fixed input normalization
into precomputed per-channel affine coefficients. It keeps the existing
bilinear rounding, RGB-to-BGR layout, detector mean/std, and recognizer
`[-1,1]` arithmetic, while removing three per-pixel floating-point divisions
from detector and crop preparation. This applies to serial OCR and true
`[N,3,H,W]` detector/recognizer batches, without adding a temporary image
buffer or changing the x86 AVX-512/AVX2/scalar/NEON runtime dispatch.

The post-change ONNX Runtime decoded-text gate passed **8/8** diverse PPM
sizes for every official model (**24/24 total**): `16x16`, English, noisy
low-contrast, `3000x80`, English/CJK single lines, mixed text, and CJK. A
four-page CPU batch smoke across the smallest, normal, low-contrast, and
ultra-wide samples remained byte-for-byte result equivalent to serial OCR
and measured **190.614 / 176.096 ms (1.082x)** serial/batched for tiny.
This explicitly exercises multiple aspect ratios rather than using a
same-page-only batch claim.

A fresh `en.ppm` three-warmup/ten-run snapshot (12 kernel workers) measured
CPU tiny **35.073 ms**, small **72.582 ms**, medium **244.095 ms**; hybrid
measured **34.887 / 72.036 / 244.847 ms**. The hybrid implementation still
selects Vulkan only after per-shape numeric validation of the complete
upload, dispatch, fence, and readback boundary; therefore it may reduce CPU
work without claiming a universal latency gain on UMA hardware. `gpu_only`
remains an explicit failure until all PP-OCRv6 operators are device-resident.

The expanded five-size memory soak (`16x16`, English, low-contrast,
`3000x80`, dense `1920x1080`) was also run twice per model. It showed no
unbounded retained activation frontier: tiny ended **14.63 MiB below** its
146,534,400-byte warm baseline; small and medium ended **8.66 MiB** and
**11.69 MiB** above their 178,892,800-byte and 425,119,744-byte baselines.
Those modest Windows allocator deltas are reported rather than hidden; a
future allocator-pool change must improve this measured dense-page result
without weakening the 24-case text-parity gate.

### 2026-08-12: dense recognizer GEMM tile admission refinement

The medium dense-page operator profile identified transformer dense projection
as the largest recurring recognizer cost (`FusedMatMulBias` was **28.685 ms**
for one profiled crop batch), ahead of any scalar postprocessing. The AVX-512
four-row GEMM micro-kernel therefore now requires both `K >= 384` **and**
`N >= 384`. This keeps it on the PP-OCRv6 medium MLP expansion
`192x768` while retaining the lower-register-pressure two-row implementation
for the paired `768x192` projection. The runtime CPU hierarchy is unchanged:
AVX-512 is selected only after runtime CPUID/XCR0 checks, then AVX2/FMA,
then portable scalar/NEON.

On the dense `1920x1080` medium UI (122 boxes, `rec_batch_size=4`,
`rec_parallelism=8`, eight kernel workers, one warm-up/three runs), the
refined default measured **4,328.474 ms**; disabling the four-row tile for
A/B gave **5,303.048 ms** (with a 6,549 ms shared-host tail). On `en.ppm`,
the same model measured **210.125 ms** versus **210.604 ms** disabled across
three warmups/ten runs. The normal AVX-512 kernel smoke, all three models'
eight-shape ONNX Runtime decoded-text gates (**24/24**), the Radeon 780M
Vulkan batch smoke, and a varied-shape hybrid batch identity check all passed.
The hybrid check continues to admit Vulkan only after complete-boundary
validation; in this sample it selected affine+Swish and depthwise variants
where GPU was faster, while keeping transfer-bound work on CPU.

### 2026-08-12: GPU-aware NCHW batch benchmark controls

`ppocr_batch_smoke` now supports repeatable steady-state measurements through
`PPOCR_BATCH_WARMUP` and `PPOCR_BATCH_RUNS`. It still verifies that every
batched page result (text, confidence, axis-aligned box, and four-point box)
matches serial inference exactly, then reports mean and median serial/batch
times. This exercises the existing real `[N,3,H,W]` detector batches and
cross-page recognizer crop batches; it is not a scheduler-only benchmark.

On the local Radeon 780M / AVX-512 host, four identical `en.ppm` tiny pages
with `det_batch_size=4`, `rec_batch_size=4`, one outer worker, two warm-ups,
and five timed runs produced eight serial-identical results. CPU measured
**192.813 / 137.304 ms** serial/batched (**1.404x**); hybrid measured
**175.306 / 125.126 ms** (**1.401x**). The hybrid result remains governed by
per-shape full H2D + dispatch + fence + D2H validation, so a GPU segment is
used only when it measured no slower than SIMD CPU for that exact batch shape.

The same hybrid configuration, with two warm-ups and three timed runs, gave
small **330.788 / 308.714 ms** (**1.072x**) and medium **991.291 / 1061.823
ms** (**0.934x**) serial/batched. The medium result is intentionally retained:
on this shared UMA host, combining page/crop work can lose to cache pressure
and transfer-bound segments. Batch support therefore does not claim a
universal GPU gain, and `gpu_only` remains unavailable until the full graph is
device-resident.

The latest run also reconfirmed dynamic high-ISA dispatch. With 12 kernel
workers and three warm-ups/six `en.ppm` runs, shape-gated AVX-512 pointwise
eight-output tiling measured tiny/small/medium **35.979 / 76.015 / 252.816
ms**. Disabling only that tile via `PPOCR_DISABLE_AVX512_POINTWISE8=1`
measured **36.286 / 81.219 / 268.576 ms**. The tile remains limited to
`C_in >= 192`, `C_out >= 192`, and plane >= 256; AVX-512 selection is runtime
CPUID/XCR0-gated, followed by AVX2 and portable scalar/NEON fallbacks.

### 2026-08-12: repeatable batch, Vulkan, memory, and model-size regression

This pass kept the production CTC executor's terminal-logit memory mode
opt-in (`PPOCR_ENABLE_FUSED_TERMINAL_CTC=1`): it replaces a full terminal
`[N,T,V]` logits activation with a per-sequence row workspace, but remains
off by default because the regular SIMD GEMM path is faster on this host.
This preserves the lower peak-memory option without making an unmeasured
throughput trade-off the default.

The complete CPU-side five-size private-memory soak was rerun with 12 kernel
workers over `16x16`, English, low-contrast, ultra-wide `3000x80`, and dense
`1920x1080` inputs. After all shapes were warmed, tiny finished cycle two
**10.09 MiB below** its **126,599,168-byte** baseline. Small and medium one
post-warm cycles reported **+4.07 MiB** from **178,348,032 bytes** and
**+10.47 MiB** from **425,025,536 bytes** respectively. These allocator
deltas are reported as observed rather than treated as a leak verdict.

The decoded-text ONNX Runtime regression passed the full eight-shape corpus
for tiny and small (**8/8 each**), and the cross-model boundary subset
(`16x16`, English, low-contrast, `3000x80`) passed medium (**4/4**). The
Radeon 780M Vulkan primitive suite also passed numerical validation. In its
latest sample hybrid admitted affine+Swish (**0.18734 vs 0.37020 ms** GPU/CPU),
depthwise+Swish (**0.33626 vs 0.61096 ms**), and depthwise+HardSwish
(**0.31810 vs 0.34620 ms**); transfer-bound projection, GEMM, ordinary Conv,
and resize work stayed on CPU automatically.

Finally, a paired six-run medium English benchmark with two warm-ups and 12
kernel workers measured pure C++ CPU **259.037 ms** versus ONNX Runtime CPU
**297.913 ms** (**13.1% lower latency**, **3.860 vs 3.357 FPS**). The ORT
sample had shared-host variation, so this is a current reproducible snapshot,
not a universal multiplier.

### 2026-08-12: optional large-map SIMD GELU admission

`PPOCR_APPROX_GELU=1` now applies the PP-OCR-specific vector GELU
approximation only to tensors with at least 65,536 values. This avoids the
short, decode-sensitive boundary activations while allowing the AVX-512/AVX2
path to amortize its approximation on genuine large feature maps. Exact
Erf-GELU remains the default, and Arm/NEON remains on the portable exact path.

With 12 kernel workers, three warm-ups, and ten English-page runs, the gated
opt-in mode measured tiny/small/medium **27.295 / 53.594 / 204.034 ms**,
versus exact-default **28.491 / 61.181 / 214.164 ms** in the paired session:
**4.2% / 12.4% / 4.7%** lower latency. The opt-in decoded-text gate passed
tiny **8/8**, small **8/8**, and medium **4/4** across the same diverse
dimensions used above. A mixed-size hybrid page/crop batch also remained
serial-identical (four pages, seven results); it measured **161.524 /
164.530 ms** serial/batched on this heterogeneous mix, so no universal batch
speedup is claimed for unlike page shapes.

### 2026-08-12: Vulkan batch revalidation

After rebuilding the Vulkan batch path, the Radeon 780M primitive smoke test
again passed numerical validation. The complete-boundary policy kept the
transfer-bound binary, pointwise, depthwise, GEMM, ordinary-convolution and
resize probes on CPU; affine+Swish was the only selected GPU primitive in the
latest pass (**0.42584 vs 0.43290 ms** GPU/CPU). These are per-primitive
measurements including H2D, dispatch, fence and D2H, not a claim that the
whole OCR graph is GPU-resident.

On four same-shape `en.ppm` pages with hybrid selected, one page worker,
`det_batch_size=4`, `rec_batch_size=4`, one warm-up and three timed runs,
the serial/batched comparison was **148.272 / 115.169 ms** (**1.287x**).
The result check remained exact at the public OCR result level (text, box and
confidence). The scheduler sends the real `N=4` tensor to both the CPU SIMD
and Vulkan primitive batch dimensions; heterogeneous inputs remain governed
by the existing width/shape buckets instead of being padded arbitrarily.

### 2026-08-12: GPU batch-path cleanup and recheck

The unvalidated experimental pointwise-to-depthwise Vulkan chain was removed.
It had never been reachable from the hybrid executor and had previously failed
its numerical validation, so retaining it would have overstated device-resident
graph support. The production Vulkan shader now contains only smoke-tested
primitive modes. `gpu_only` therefore continues to reject execution explicitly;
the implementation must not silently fall back to CPU until the complete
PP-OCRv6 graph, DB post-processing, and CTC decode can remain on the device.

After the cleanup, the Radeon 780M Vulkan primitive smoke and AVX-512 kernel
smoke both passed. The same four-page tiny English batch (`det_batch_size=4`,
`rec_batch_size=4`, one outer worker, one warm-up, three timed runs) remained
serial-identical: CPU was **140.573 / 100.661 ms** serial/batched (**1.396x**),
and hybrid was **135.904 / 99.425 ms** (**1.367x**). A hybrid ONNX Runtime
text gate also passed **4/4** diverse tiny inputs (16x16, English,
low-contrast, and 3000x80 ultra-wide). These figures exercise the public
real-N detector and recognition batches; they do not claim a complete
GPU-resident inference path.

### 2026-08-12: fused approximate BatchNorm-GELU

When the existing opt-in `PPOCR_APPROX_GELU=1` mode is enabled, a large
fused BatchNorm-GELU group now performs its channel affine and AVX-512/AVX2
Pad鑼?GELU approximation in one vector pass. This removes the previous
per-plane affine write/read round trip. The normal exact-Erf path is unchanged;
non-x86/NEON builds also retain its exact behavior. Set
`PPOCR_DISABLE_FUSED_BN_GELU=1` only to reproduce the former two-pass
experimental implementation for A/B measurement.

On the local AVX-512 host with 12 worker threads, two warm-ups and six English
runs, the fused mode measured tiny **27.106 ms**, small **51.901 ms**, and
medium **225.597 ms**. The former two-pass opt-in path measured **27.241 / 
62.450 / 232.207 ms** respectively, a material small-model reduction and a
positive medium improvement in that paired session. Decoded-text regression
against ONNX Runtime passed the eight-size corpus for tiny, small, and medium
(**24/24** total): 16x16, English, low-contrast, ultra-wide, English/CJK
single-line, mixed text, and CJK. The broad requirement to outperform ONNX
Runtime on every model and workload remains a continuing target, not a claim
from this one host-local optimization sample.

### 2026-08-12: batch-invariant GELU admission correction

The optional SIMD GELU admission was tightened so batching cannot change a
page's numerical execution path. Generic GELU is now always exact, because it
has no NCHW shape information and a total-element threshold could make a
singleton crop use Erf while the same crop in a batch used the approximation.
The model-aware fused BatchNorm+GELU path continues to use its AVX-512/AVX2
one-pass approximation only when each NCHW plane is at least 65,536 elements;
this predicate is invariant to the leading batch dimension. Exact GELU remains
the default and ARM/NEON retains the exact fallback.

After the correction, kernel and Radeon 780M Vulkan smoke both passed. The
hybrid decoded-text ONNX Runtime gate passed **4/4** tiny diverse inputs
(16x16, English, low contrast, and 3000x80), with `PPOCR_APPROX_GELU=1`.
The public four-page `en.ppm` batch gate also restored exact serial/batch
results at `det_batch_size=4`, `rec_batch_size=4`, and one recognition worker:
**169.174 / 123.454 ms (1.370x)** for its three-run hybrid sample. A longer
ten-run local snapshot measured CPU **165.446 / 118.013 ms (1.402x)** and
hybrid **210.649 / 142.646 ms (1.477x)**. These shared-host figures are batch
throughput evidence, not an end-to-end GPU-only performance claim; hybrid
still admits only individually validated full-boundary Vulkan segments.

### 2026-08-12: MatMul write-only activation allocation

The constant-weight transformer MatMul path now allocates its destination with
`resize()` rather than value-initializing it to zero. `kernels::Gemm()` writes
every output element from its bias/zero accumulator, so the previous clear was
dead memory traffic before the SIMD or Vulkan attempt. This reduces activation
write bandwidth and allocator work for the recognizer's tall batch-folded
projections without changing the CPU, hybrid, or ARM/NEON fallback semantics.

Post-change kernel and Vulkan smoke passed, including the Radeon 780M
full-boundary admission checks. Hybrid decoded-text parity against ONNX Runtime
passed the eight-size corpus for every supported model (**24/24**): 16x16,
normal English, low contrast, 3000x80, English/CJK lines, mixed text, and CJK.
A separate eight-image mixed-aspect tiny hybrid batch check returned all 14
serial-identical OCR results at **264.126 / 256.614 ms (1.029x)**; this is a
diversity/correctness gate and is intentionally not represented as a universal
throughput claim.

For a current 12-worker, three-warmup/twenty-run `en.ppm` CPU comparison on
this shared host, the measured C++ / ONNX Runtime means were tiny **34.049 /
85.073 ms** (60.0% lower C++ latency), small **75.880 / 106.510 ms** (28.8%
lower), and medium **249.121 / 210.750 ms** (18.2% higher). The medium sample
does not satisfy the project performance target and is recorded explicitly:
the remaining work is medium recognizer/detector optimization and longer
device-resident Vulkan segments, not a claim that this pass has already made
all models faster than ONNX Runtime.

### 2026-08-12: multi-adapter Vulkan selection for batched hybrid execution

The Vulkan backend now evaluates all compute-capable adapters instead of
blindly selecting Vulkan's first enumerated queue. Its default selection is
the integrated/UMA adapter, which is the appropriate default for the current
per-segment H2D/dispatch/D2H hybrid design; set
`PPOCR_VULKAN_PREFER_DISCRETE=1` to prefer a discrete accelerator. Deployment
integration can pin an adapter with `PPOCR_VULKAN_DEVICE_INDEX=N` or select it
by substring through `PPOCR_VULKAN_DEVICE_NAME=...`. CPU Vulkan drivers are
not chosen automatically, but may still be requested by index for diagnostic
purposes. Every selected operator remains gated by its exact shape/batch,
full-boundary no-slower-than-CPU measurement.

After this change, the Radeon 780M Vulkan smoke suite and AVX-512 kernel smoke
both passed. A four-page same-shape tiny batch (`det_batch_size=4`,
`rec_batch_size=4`, hybrid, one page worker, two warmups/five timed runs)
returned all eight public OCR results identically. The shared Windows host
showed unstable timing in the immediate post-build sample (**170.452 ms serial
/ 219.688 ms batch; batch median 119.245 ms**), so no new throughput claim is
made from that run. It nevertheless validates real `[N,3,H,W]` detector and
crop batches on the selected GPU path; it does not represent a GPU-only claim
because the complete graph, DB post-processing, and CTC decode are still not
device-resident.

### 2026-08-12: GPU batch front-end parallelism

The public batched API now exposes `Options::batch_preprocess_parallelism`
(default **4**), with the benchmark override
`PPOCR_BATCH_PREPROCESS_PARALLELISM`. For a same-shape detector batch, each
page's resize/BGR-normalize write owns a separate contiguous NCHW plane and is
prepared concurrently before the one batched CPU/Vulkan graph submission.
The budget is divided across concurrent dynamic-shape detector batches, so
raising `image_batch_parallelism` does not cause unbounded nested-thread
oversubscription. It is CPU front-end work that enables GPU/SIMD batching; it
does not change the strict Vulkan transfer/admission policy or imply a
GPU-only graph.

On the local Radeon 780M / AVX-512 host with the tiny model, four identical
`en.ppm` pages, `det_batch_size=4`, `rec_batch_size=4`, one page worker,
`PPOCR_APPROX_GELU=1`, three warm-ups, and ten timed hybrid runs, the
serial-identical public result check passed in both configurations. Serial
preprocess (`PPOCR_BATCH_PREPROCESS_PARALLELISM=1`) measured **121.887 ms**
batched mean (**1.479x** against **180.328 ms** serial); the default four-way
front end measured **119.919 ms** (**1.498x** against **179.646 ms** serial).
This is a **1.6%** improvement in that controlled local batch sample. Timing
on the shared Windows host remains variable, so the README records it as a
reproducible configuration-specific result rather than a universal GPU gain.

### 2026-08-12: experimental Vulkan tiled GEMM+Swish batch kernel

The Vulkan shader now includes an 8-row x 128-column cooperative GEMM tile
with Swish fused into its final writeback (mode 25). It shares a 16x128 weight
tile across eight flattened recognition rows, so it is designed for real
same-width crop batches rather than per-crop dispatches. Its FP32 K reduction
remains ascending, and the shader stays behind the existing numerical smoke
and complete H2D + dispatch + fence + D2H admission checks. Enable it only for
driver-specific A/B work with `PPOCR_ENABLE_VULKAN_GEMM_SWISH_TILE=1`; the
plain tiled GEMM path remains dynamically selected as before.

On the local Radeon 780M, AVX-512 host, the experimental tile passed kernel
and Vulkan primitive smoke plus the hybrid ONNX Runtime decoded-text gates for
tiny and medium (4/4 diverse inputs each). A quick medium four-page A/B saw
the tile at **1004.444 ms** batched mean and the default at **1041.713 ms**;
the longer shared-host samples varied enough that neither result is promoted
to the default. The default public medium batch remained result-identical and
measured **1178.757 / 1061.702 ms** serial/batched (**1.110x**) with two
warm-ups and six timed runs. This is an implementation and correctness
milestone, not a claim of a universal GPU speedup; hybrid continues to select
GPU only for primitives that win their exact full-boundary admission
measurement.

### 2026-08-12: recognition-batch front-end parallelism

`batch_preprocess_parallelism` now also prepares same-width recognition crops
in parallel. Each resize owns one already-zeroed, contiguous `[N,3,48,W]`
plane; output order, natural resize width, right padding, and normalization
arithmetic are unchanged. The front-end budget is divided by concurrent
recognizer batches, avoiding nested oversubscription when
`rec_parallelism > 1`.

On the local Radeon 780M / AVX-512 host, four same-shape tiny English pages
with hybrid mode, `det_batch_size=4`, `rec_batch_size=4`, one recognizer
worker, two warm-ups and eight timed runs were public-result identical. A
serial front end measured **112.023 ms** batch mean; four-way front-end work
measured **105.894 ms**, a **5.5%** batch-latency reduction. The corresponding
serial/batch speedups were **1.523x / 1.577x**. A five-run small-model check
remained result-identical too (**314.876 / 313.329 ms** batch at one/four-way
preprocess); it is kept as evidence that the gain is input/model dependent,
not as a universal throughput claim. Kernel and Vulkan primitive smoke passed
after the change; decoded-text ONNX Runtime checks also passed the small
model's 16x16, English, low-contrast and ultra-wide inputs. Very dense/tall
synthetic pages remain a deliberately stricter diagnostic corpus: their
near-threshold DB boxes can differ from ORT by one pixel and therefore must
not be used as a text-parity gate without a geometric tolerance.

### 2026-08-12: GPU-ready single-page crop batching

The same crop-plane parallelism is now used by both public entry points:
`RecognizeBatch()` and ordinary `Recognize()`. A dense single page can contain
several same-width recognition crops, so this closes a scheduling gap where
only multi-page batches could prepare their real `[N,3,48,W]` input planes in
parallel. The budget is `Options::batch_preprocess_parallelism` (or
`PPOCR_BATCH_PREPROCESS_PARALLELISM`) and is divided by active recognizer
batches, which prevents nested CPU oversubscription. Each worker owns a
pre-zeroed NCHW plane; width padding and every pixel/normalization calculation
remain unchanged.

This benefits the existing CPU SIMD and Vulkan hybrid paths equally. Vulkan
continues to receive a genuine NCHW batch, with its batch dimension mapped to
the shader dispatch Y dimension. It still offloads only an operator after its
same-shape, full H2D + dispatch + fence + D2H numerical/performance admission
test is no slower than the CPU kernel; `gpu_only` remains unavailable because
the complete PP-OCRv6 graph, DB postprocessing, and CTC decode are not
device-resident.

Post-change verification passed the AVX-512 kernel smoke, Vulkan primitive
smoke, four-page public result identity, and a medium varied-shape hybrid
memory soak. On the local Radeon 780M / AVX-512 host, four equal `en.ppm`
tiny pages in hybrid mode (`det_batch_size=4`, `rec_batch_size=4`, one page
and recognizer worker, two warm-ups, six timed runs) measured **117.702 ms**
with serial front-end work and **110.992 ms** with a four-way front end: a
**5.7%** batch-latency improvement (serial/batch speedup **1.420x ->
1.498x**). A short medium CPU batch check also preserved all eight public OCR
results and measured **1118.404 / 1021.506 ms** serial/batched (**1.095x**).
These shared-host figures are configuration-specific benchmarks, not a claim
that every model or GPU segment is faster.

### 2026-08-12: experimental AVX-512 wide-context tile

Medium PP-OCRv6 detector context blocks include padded 5x5 and 7x7
convolutions. An experimental AVX-512 eight-output tile now shares each
16-float input load across eight output channels, while keeping the existing
four-output implementation for a non-multiple-of-eight tail. It preserves
each output's increasing input/channel/kernel FMA order and has no effect on
AVX2, ARM/NEON, or scalar dispatch. Enable it only for deployment A/B work
with `PPOCR_ENABLE_AVX512_CONV_WIDE8=1`; the established four-channel tile
remains the default until it wins reliably across target CPUs.

Both default and opt-in paths passed the kernel smoke, including padded 5x5
and 7x7 eight-channel bodies plus nine-channel tails, and the full Radeon
780M Vulkan primitive suite. On this shared AVX-512 host, a medium `en.ppm`
CPU sample (two warm-ups/eight runs, twelve workers) measured **223.342 ms**
default and **221.962 ms** opt-in (0.6% lower mean). A 122-crop
`ui_dense_1920x1080` sample did not reproduce that gain (**4053.814 vs
4075.517 ms**, two warm-ups/five runs), so this remains deliberately opt-in
rather than a broad performance claim. Hybrid Vulkan admission remains
unchanged: every GPU primitive is still selected only when its validated full
H2D + dispatch + fence + D2H boundary is no slower than CPU.

The same post-change CPU 12-worker `en.ppm` snapshot (two warm-ups/ten runs)
was tiny **36.902 ms**, small **72.035 ms**, and medium **272.094 ms**.
Hybrid runs (two warm-ups/six runs) were **37.502**, **72.662**, and
**255.454 ms** respectively. The medium hybrid improvement is useful on this
UMA GPU, while CPU remains slightly better for tiny/small in this run; the
per-primitive admission rule intentionally preserves that outcome instead of
forcing Vulkan. Medium decoded-text parity against ONNX Runtime also remained
**4/4** across 16x16, English, low-contrast, and 3000x80 inputs with the
experimental wide tile enabled. These measurements do not change the prior,
more representative ONNX Runtime comparison: medium has not yet demonstrated
a consistent end-to-end lead over ORT and remains the main optimization target.

### 2026-08-12: fused GPU ordinary-Conv activation batches

The hybrid Vulkan ordinary NCHW convolution primitive now also accepts
graph-fused **Swish**, Sigmoid, and HardSwish writebacks. Non-pointwise
detector/recognizer Conv activation chains therefore use one batched GPU
submission, with no GPU-to-CPU readback merely to apply the nonlinear pass.
The CPU SIMD fallback is unchanged. Admission remains exact per
`(N,C,M,H,W,kernel,stride,padding,activation)` and includes H2D, dispatch,
fence, D2H, and output validation against CPU Conv plus the same activation.
If this complete boundary loses, hybrid keeps CPU; `gpu_only` remains
unavailable.

`ppocr_vulkan_smoke` now prints an independent `Conv+Swish` batch admission
measurement next to its existing `Conv+ReLU` result. This is primitive-level
evidence only; an end-to-end OCR GPU claim requires the corresponding shapes
to pass their dynamic admission measurements.

Post-change AVX-512 kernel smoke and Radeon 780M Vulkan smoke both passed.
The new ordinary `N=4,C=24,M=40,31x29,3x3` Swish probe measured **3.58512 ms
GPU vs 1.24636 ms CPU**, so hybrid correctly retained CPU for that segment on
this UMA device. A hybrid four-page exact-shape batch regression also remained
result-identical: tiny measured **109.844 / 84.810 ms** serial/batched
(**1.295x**), and medium **920.012 / 836.339 ms** (**1.100x**) with two
warm-ups and three timed runs, `det_batch_size=4`, `rec_batch_size=4`, and
four-way preprocessing. Medium decoded-text ONNX Runtime parity passed **4/4**
for 16x16, English, low-contrast, and ultra-wide inputs. These are controlled
batch measurements on a shared host, not a GPU-only or universal throughput
claim.

### 2026-08-12: liveness-aware in-place Concat

Both graph executors now reuse the first dynamic input of a final-use ONNX
`Concat` as the output storage. The general implementation handles every
legal axis: after growing the vector, it moves the first input's outer blocks
backwards (overlap-safe) and copies only the additional branch spans. This is
especially relevant to the medium detector's channel-axis FPN assembly, where
operator profiling previously exposed `Concat` as a measurable cost. It avoids
one complete output allocation and the copy of the first branch while keeping
the generic path for shared/aliased/non-final-use inputs. `RunCtcTop1()` has
the same rule, so public recognizer batches do not diverge from `Run()`.

Verification after the change passed AVX-512 kernel smoke, Radeon 780M Vulkan
smoke, and medium CPU decoded-text ONNX Runtime parity on 16x16, English,
low-contrast, and 3000x80 inputs (**4/4**). A medium hybrid seven-size memory
soak (16x16, English, low contrast, ultra-wide, ultra-tall, dense 1920x1080,
and 2560x1440 screenshot) finished one cycle at **+14,647,296 bytes** over a
453,398,528-byte post-warmup private-byte baseline. This is allocator
high-water behavior, not a leak assertion. The immediate six-run medium
`en.ppm` CPU sample measured **234.826 ms**; hybrid measured **258.733 ms** on
this shared UMA host, so the dynamic per-segment rule correctly does not claim
a blanket GPU gain.

### 2026-08-12: hybrid GPU batching and queue-aware scheduling

The Vulkan path continues to accept true detector and recognition NCHW
batches, and every implemented primitive uses the leading `N` dimension as
one dispatch dimension. This includes pointwise/residual, depthwise and
ordinary Conv, ConvTranspose, resize+Add, channel-affine, binary chains, and
recognizer GEMM. Immutable model weights/biases remain in the persistent
Vulkan context when an admitted segment repeats; dynamic images and residuals
are uploaded once per complete batch. The runtime restores compact descriptor
views before every dispatch, preventing a preceding cached parameter shape
from affecting a later batched operator.

Hybrid execution has one synchronized Vulkan queue, so parallel host graph
batches cannot run device work concurrently. `Options::hybrid_graph_parallelism`
therefore defaults to **1** (benchmark override
`PPOCR_HYBRID_GRAPH_PARALLELISM`), avoiding serialized submissions together
with multiplied CPU activation workspaces. CPU-only keeps
`rec_parallelism` independent batches in flight. This is a scheduling and
memory-peak safeguard, not a restriction on the tensor batch itself; use a
positive override only after measuring the target driver.

After the change, the AVX-512 kernel smoke and Radeon 780M Vulkan primitive
smoke both passed. On four same-shape tiny `en.ppm` pages with hybrid mode,
`det_batch_size=4`, `rec_batch_size=4`, four-way preprocessing, one warm-up,
and three timed runs, public OCR results were serial-identical. The queue-aware
default measured **150.969 / 105.663 ms** serial/batched (**1.429x**). An
explicit four-graph-worker A/B measured **131.838 / 102.749 ms** (**1.283x**)
on the same shared host. CPU-only under the same batch settings measured
**133.062 / 98.406 ms** (**1.352x**). These short, shared-host snapshots
demonstrate correct batch scheduling; they do not alter the strict
per-primitive GPU admission policy, do not claim universal GPU speedups, and
do not make `gpu_only` available.

### 2026-08-12: refreshed three-model and varied-size baseline

`ppocr_mem_soak` now accepts the same backend and bounded-batch environment
settings as the benchmark (`PPOCR_BACKEND`, detector/recognition batch sizes,
preprocessing, and hybrid graph parallelism). This makes the memory check
cover the production CPU/hybrid scheduler rather than always constructing a
default OCR instance.

With `PPOCR_APPROX_GELU=1`, CPU mode, two warm-ups and eight timed `en.ppm`
runs on this AVX-512 host, current C++ means were tiny **28.602 ms**, small
**58.717 ms**, and medium **206.668 ms**. Hybrid on the same short workload
measured **32.243 / 70.071 / 259.296 ms** respectively, so CPU was correctly
retained for this particular small-map run when Vulkan segment admission did
not improve the full OCR boundary. The earlier representative ORT comparison
still establishes a lead for tiny/small; medium remains an active optimization
target and is not claimed universally ahead of ONNX Runtime.

A medium hybrid varied-size one-cycle soak with seven samples (16x16,
English, low-contrast, ultra-wide, ultra-tall, dense 1920x1080, and
2560x1440 screenshot), `det_batch_size=4`, `rec_batch_size=4`, and four-way
preprocessing finished at **+2,134,016 bytes** over a
**436,551,680-byte** post-warmup private-byte baseline. This is a process
allocator high-water snapshot, not a leak proof; it demonstrates the current
test corpus and bounded activation behavior across materially different input
dimensions.

### 2026-08-12: queue-aware detector batching

`RecognizeBatch()` now applies the same queue-aware graph scheduling rule to
the detector as it already applied to recognition. `image_batch_parallelism`
still controls independent detector graphs in CPU mode. In hybrid mode, the
default is instead `hybrid_graph_parallelism=1`: one persistent synchronous
Vulkan queue cannot execute competing detector graphs concurrently, and
running several of them only retains several large detector activation
frontiers while their GPU submissions serialize. This does not reduce the
real tensor batch: same-resize pages still form bounded `[N,3,H,W]` detector
batches and all implemented Vulkan kernels map that N dimension to one
dispatch. Set `PPOCR_HYBRID_GRAPH_PARALLELISM` above one only after a
driver-specific A/B measurement.

Post-change AVX-512 kernel and Radeon 780M Vulkan primitive smoke both
passed. Four identical `en.ppm` pages, `det_batch_size=4`,
`rec_batch_size=4`, four-way preprocessing, two warm-ups and five timed runs
were serial-result-identical. With the hybrid default, tiny measured
**147.010 / 107.388 ms** serial/batched (**1.369x**) and medium measured
**1050.363 / 1026.150 ms** (**1.024x**). The matching CPU-only measurements
were **136.004 / 105.275 ms** (**1.292x**) for tiny and **994.335 / 976.100
ms** (**1.019x**) for medium. An explicit hybrid four-graph-worker experiment
was not a win in this sample: tiny batched latency rose to **158.929 ms**;
medium rose to **1096.940 ms**. A four-image varied-size medium hybrid smoke
also stayed serial-result-identical at **1242.143 / 1205.323 ms** (**1.031x**).
These shared-host measurements validate the scheduler and batch correctness;
they do not claim a universal GPU advantage. `gpu_only` remains unavailable
until the full PP-OCRv6 graph and post-processing are device-resident.

### 2026-08-12: medium CPU parallelism retune

The persistent CPU kernel executor's automatic ceiling is now **16** workers
(bounded by available hardware); `PPOCR_BENCH_THREADS` remains the explicit
deployment override. The previous automatic ceiling of 12 left measurable
medium-detector channel/tile parallelism unused on this AVX-512 host. Tiny and
small keep their short-kernel thresholds, so this is not a promise that every
model launches sixteen workers.

On the local AVX-512 host with `PPOCR_APPROX_GELU=1`, two warm-ups and twelve
timed `en.ppm` runs, the default CPU path measured tiny **28.124 ms**, small
**61.297 ms**, and medium **188.145 ms**. A medium 3000x80 strip measured
**130.692 ms** over eight runs. The controlled medium A/B was 12 workers:
**215.234 ms** mean, 16 workers: **189.517 ms** mean (both twelve-run
samples); 20 and 24 workers regressed, so the automatic cap is deliberately
16 rather than unbounded. The current same-host ORT eight-run `en.ppm`
snapshot is tiny **74.609 ms**, small **86.288 ms**, and medium **147.779
ms**: tiny/small retain clear native wins, while medium remains slower than
ORT on that comparison and is not claimed as meeting the final target.

The latest Radeon 780M Vulkan primitive suite still passed. Its full-boundary
admission selected GPU for the Swish channel-affine and depthwise-Swish batch
shapes, while ordinary Conv, pointwise Conv, GEMM and resize/add samples
remained transfer-bound and correctly retained CPU. A medium hybrid `en.ppm`
sample was **295.677 ms** (eight runs), versus the CPU result above, confirming
that a device-resident multi-operator graph segment閳ユ攺ot forced standalone
dispatches閳ユ攰s required before claiming a medium GPU end-to-end win.

### 2026-08-12: N=8 GPU-aware page batch and PPM stress gate

The public batch route was rechecked with eight identical tiny `en.ppm`
pages. It forms an exact `[8,3,H,W]` detector tensor and packs compatible
recognition crops into `[N,3,48,W]`; no page or crop is resampled merely to
fill a batch. Vulkan maps that leading N dimension to one compute-dispatch
dimension for each implemented hybrid primitive. With 16 CPU kernel workers,
`det_batch_size=8`, `rec_batch_size=8`, eight-way preprocessing, one graph
submitter, two warm-ups and eight timed runs, serial results and batch results
were exactly identical (16 decoded OCR results):

| Backend | Serial mean | N=8 batch mean | Batch speedup |
|---|---:|---:|---:|
| CPU | 337.102 ms | 233.977 ms | 1.441x |
| Hybrid Vulkan | 454.287 ms | 239.397 ms | 1.898x |

This is a same-host batch scheduling measurement, not a claim that the
partial Vulkan graph makes whole-image OCR faster than CPU on the Radeon 780M:
the hybrid N=8 mean was still 2.3% above the CPU N=8 mean. It does verify that
Vulkan-enabled batching is a real single-NCHW submission path with correct
output, rather than eight independent GPU calls.

### 2026-08-13: front-end address reduction and expanded recovery check

The fused RGB bilinear-resize/NCHW-normalization front end now hoists its two
source-row bases out of the destination-pixel loop.  This removes four
row/column address multiply-add chains per output pixel, without changing the
four source samples, interpolation order, integer rounding, or model input.
It applies to both detector-page resize and recognizer-crop resize paths, so
the benefit is architecture-neutral and composes with the existing AVX2,
AVX-512, and NEON graph kernels.

The change passed the complete low-level SIMD smoke suite and decoded-text
comparison with ONNX Runtime for all model sizes: tiny, small, and medium each
passed **4/4** inputs (`16x16`, English two-line, low-contrast text, and an
ultra-wide `3000x80` strip), for **12/12** total.  The wider robustness corpus
also passed **11/11** cases for tiny and small: blank, word, rotated text,
low-contrast text, wide/tall strips, dense blobs, dense UI, and 1920x1080 /
2560x1440 screenshots.  Medium passes the same semantic corpus except its
current detector threshold produces extra low-score components on the blank
and dense-blob synthetic cases; this is tracked as DB-postprocess tuning work,
not hidden as a performance success.

A fresh exact `N=4` CPU batch recheck (`det_batch_size=4`,
`rec_batch_size=4`, four-way preprocessing, one warm-up, five timed runs)
kept all eight decoded outputs serial-identical.  Mean batch speedups on this
shared AVX-512 Windows host were tiny **1.490x**, small **1.129x**, and medium
**1.243x**.  The medium value has visibly variable shared-host samples, so
these figures are evidence of the batched code path rather than portable
capacity guarantees.

The expanded `16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720
-> 1920x1080 -> 2560x1440 -> 64x64` memory ladder was rerun for medium.  CPU
passed two cycles from a 146,903,040-byte warmed private baseline and returned
to 143,212,544 bytes (-2.51%) and 144,183,296 bytes (-1.85%); private bytes
and working set both declined after the largest image in both cycles.  Hybrid
Vulkan also passed: its 392,998,912-byte baseline returned to 393,195,520
bytes (+0.050%) in both cycles.  Hybrid private bytes remained 0.0031% above
the immediately preceding largest sample while working set declined, which is
expected persistent Vulkan/Windows allocator capacity rather than retained
image or activation memory.

### 2026-08-13: selective SIMD GELU dispatch

The opt-in `PPOCR_APPROX_GELU=1` path now reaches independent generic
`FusedGelu` activations as well as the existing fused BatchNorm+GELU route.
On x86 it dynamically selects AVX-512 first, then AVX2; the baseline exact
Erf implementation and ARM/NEON fallback remain unchanged.  The approximate
Pad鑼?tanh polynomial is intentionally limited to broad activations of at
least 262,144 elements.  That preserves the established exact path for the
short, decision-sensitive tiny/small recognizer tensors while accelerating
the large medium feature maps. `PPOCR_DISABLE_APPROX_GELU_SIMD=1` is a direct
deployment A/B escape hatch.

The AVX2 generic GELU kernel was also made genuinely vectorized: it no longer
stores each vector and calls scalar `tanh` eight times. AVX2 and AVX-512 now
use the same bounded Pad鑼?approximation for vector bodies and scalar tails.
The low-level smoke verifies separate and in-place large/tail GELU buffers,
and passed in both approximate and default exact modes.

With 16 workers on the local AVX-512 host, a paired medium CPU `en.ppm` A/B
(two warm-ups, eight timed runs) improved from **216.867 ms** with the new
generic SIMD route disabled to **205.715 ms** enabled (**5.1% lower** mean
latency).  Tiny, small and medium each again passed all four decoded-text
comparisons against ONNX Runtime (`16x16`, English, low contrast and
`3000x80`), **12/12** total.  The same-host 12-run CPU snapshot after the
change was tiny **34.335 ms** and small **75.072 ms**; shared-host variance
means these are compatibility snapshots, not portable throughput guarantees.

### 2026-08-13: SIMD terminal-CTC memory route

`PPOCR_ENABLE_FUSED_TERMINAL_CTC=1` now computes each reusable terminal-logit
row through the normal runtime-dispatched GEMM kernel instead of a separate
scalar `K鑴砎` loop. It still avoids materializing the full terminal `[N,T,V]`
logits tensor, but now uses AVX-512, AVX2, or ARM/NEON GEMM as available and
uses the existing ISA argmax. The K-order, selected-class Softmax calculation,
blank/repeat CTC recurrence, and output confidence remain unchanged.

With this opt-in mode and 16 CPU workers, all three models passed the four
diverse ONNX Runtime decoded-text inputs (**12/12**). The new N=4 public batch
smoke was serial-result-identical and measured **1.367x** (tiny), **1.149x**
(small), and **1.188x** (medium) batch speedup on this shared AVX-512 Windows
host. Two warm-ups/eight English-page runs measured **35.933 / 74.952 /
246.148 ms** for tiny/small/medium. The medium two-cycle
`16x16 閳?閳?閳?2560x1440 閳?64x64` CPU memory ladder also passed: a
**147,066,880-byte** warmed baseline returned to **143,347,712 bytes**
(-2.53%) and **147,116,032 bytes** (+0.03%), both below their pre-return
cycle high-water marks. The regular full-terminal-logits SIMD route remains
the default; this stays an explicit memory/performance deployment option.

### 2026-08-13: four-row terminal-CTC GEMM tile

The optional terminal-CTC route now projects up to four adjacent time steps
into its bounded vocabulary workspace at once, then performs the CTC scan in
the original time order. This keeps memory bounded to `4 鑴?vocab` values per
active sequence rather than `[N,T,V]`, while allowing the AVX-512/AVX2 GEMM
microkernels to reuse immutable projection weights across rows. The final
partial tile and every blank/repeat transition retain the same semantics.

On the 16-worker AVX-512 host, two warm-ups/eight `en.ppm` runs with
`PPOCR_ENABLE_FUSED_TERMINAL_CTC=1` measured **32.647 / 70.079 / 226.250 ms**
for tiny/small/medium. Compared with the preceding one-row fused-CTC sample,
this is **9.1% / 6.5% / 8.1%** lower latency. The post-change N=4 CPU smoke
remained serial-result-identical for all models and measured **1.350x**,
**1.201x**, and **1.195x** batch speedup respectively. The low-level kernel
smoke, all three four-image ONNX Runtime decoded-text comparisons (**12/12**),
and Radeon 780M Vulkan primitive/admission smoke passed. As intended, hybrid
continues to select GPU only for shapes that win the full transfer boundary;
the terminal CTC tile is a CPU-memory optimization, not a GPU-only path.

### 2026-08-13: four-row terminal-CTC resolution-ladder memory recheck

The deterministic `16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 ->
1280x720 -> 1920x1080 -> 2560x1440 -> 64x64` ladder was rerun after the
four-row terminal-CTC tile, with `PPOCR_BENCH_THREADS=16`,
`PPOCR_APPROX_GELU=1`, and `PPOCR_ENABLE_FUSED_TERMINAL_CTC=1`. The CPU
medium model passed the two-cycle bounded-recovery gate. Its warmed private
baseline was **143,339,520 bytes**; the post-large `64x64` samples were
**148,660,224 bytes** (**+3.71%**) and **147,742,720 bytes** (**+3.07%**).
They were only **+0.68%** above the immediately preceding `2560x1440`
sample in each cycle, well below the 20% gate.

In this particular shared-Windows-host run, neither private bytes nor working
set immediately declined after the largest page: the allocator retained or
reused capacity, rather than keeping the RGB image or activation objects
alive. The checker records this explicitly as `declined_after_largest=false`
and retains every per-resolution private-byte/working-set sample in
`build/memory_ladder_medium_fused_ctc_tile4_cpu_20260813.json`; it therefore
does not mislabel stable allocator capacity as immediate memory release. The
completed image is destroyed before every sample, and the bounded-recovery
pass is the portable regression condition.

### 2026-08-13: AVX-512 reconstruction 2x2 four-output default

The CPU dispatcher now enables the existing AVX-512 four-output tile for
valid `2x2` reconstruction convolutions by default when the shape has at
least eight input channels. It reuses each 16-float input vector across four
output filters while retaining the same per-output reduction order, scalar
tail, AVX2, NEON, and baseline fallbacks. This covers the tiny/small/medium
detector reconstruction pairs (`8閳?6`, `12閳?4`, and `32閳?4` channels) without
introducing an ISA requirement. Set `PPOCR_DISABLE_AVX512_CONV2X2_TILE4=1`
for a deployment A/B fallback.

On the local shared AVX-512 Windows host (`PPOCR_BENCH_THREADS=16`, opt-in
approximate GELU and four-row terminal CTC), the medium English page improved
from **237.935 ms** to **230.035 ms** across 16 timed runs (**3.3% lower**
mean latency). A two-run dense `1920x1080` sample also reduced from
**4,740.695 ms** to **4,208.679 ms** (**11.2% lower**); this dense figure is
intentionally short and host-variable, so it is directional rather than a
portable throughput guarantee. A fresh 2-warmup/8-run CPU snapshot measured
tiny **35.970 ms**, small **79.078 ms**, and medium **235.396 ms** on `en.ppm`.

All three models passed the four varied ONNX Runtime decoded-text cases
(**12/12**: `16x16`, English, low contrast, and `3000x80`) with the new
default. The N=4 public batch smoke stayed serial-result-identical and
measured **1.314x**, **1.210x**, and **1.087x** speedup for tiny, small, and
medium. The low-level exact/approximate SIMD smoke and the Radeon 780M Vulkan
admission smoke also passed. A current short hybrid sample was **35.627 /
84.529 / 294.957 ms** (tiny/small/medium), so the transfer-inclusive admission
rule continues to select Vulkan only for segments that actually meet the
no-slower condition; this does not claim complete GPU-only PP-OCRv6 support.

### 2026-08-13: terminal-projection SIMD coverage expansion

The dependency-free kernel smoke now includes the exact terminal projection
geometries of every shipped recognizer: `4x80x6906` for tiny and
`4x120x18710` / `4x192x18710` for small/medium, plus the bounded terminal-CTC
route. This explicitly exercises wide vocabulary tails, bias initialization,
the AVX-512 four-row GEMM reuse tile, and the unchanged scalar/AVX2/NEON
fallback contract rather than inferring coverage from a smaller synthetic
matrix.

A renewed CPU A/B on the English page (three warm-ups, 12 runs, 16 workers,
exact terminal logits) measured tiny/small/medium at **28.508 / 62.702 /
192.242 ms** with the default reconstruction tile, versus **28.950 / 63.142 /
198.772 ms** with `PPOCR_DISABLE_AVX512_CONV2X2_TILE4=1` (**1.5% / 0.7% /
3.3% lower**). Separately disabling the already-default AVX-512 four-row
GEMM tile produced **30.019 / 63.584 / 196.993 ms**, confirming that its main
measured benefit is medium's wide transformer projections. These remain
shared-host snapshots, not cross-machine guarantees.

### 2026-08-13: batched AVX-512 2x2 reconstruction tile

`Conv2dBatch` now uses the same AVX-512 four-output `2x2` valid-convolution
tile as the single-image dispatcher. Its scheduler owns one
`[batch, output-channel-tile]` task instead of one output plane, preserving
the exact per-channel reduction order while reusing every loaded input vector
over four filters for detector batches. The existing AVX2, NEON, scalar, tail,
and `PPOCR_DISABLE_AVX512_CONV2X2_TILE4=1` fallback paths remain intact.

With four equal English pages, `det_batch_size=4`, `rec_batch_size=4`,
four-way preprocessing, two warm-ups, and five timed runs, serial-vs-batch
results remained bit-identical. The tile-enabled batch means were **91.918 /
213.696 / 781.774 ms** for tiny/small/medium, versus **93.346 / 223.177 /
939.356 ms** with the tile disabled: **1.5% / 4.2% / 16.8% lower** batch
latency on this shared AVX-512 host. The corresponding complete
serial-to-batch speedups were **1.290x / 1.194x / 1.096x**. All three models
also passed their four-image ONNX Runtime decoded-text checks (**12/12**).

### 2026-08-13: refreshed small-to-large memory ladder

The native memory regression now exercises one live medium PP-OCRv6 instance
over a deterministic ascending image sequence: `16x16 -> 64x64 -> 160x120 ->
320x240 -> 640x480 -> 1280x720 -> 1920x1080 -> 2560x1440 -> 64x64`.  Each
image is decoded, inferred, destroyed, and only then sampled.  The report
therefore records process private bytes and Windows working-set bytes for
every resolution, plus whether the final small image returns below the
pre-large high-water mark and whether it immediately declines from the
preceding largest-image sample.

A fresh CPU run with 16 workers and `PPOCR_APPROX_GELU=1` passed two cycles
from a **143,192,064-byte** warmed private baseline.  The final `64x64`
samples were **148,197,376 bytes** (+3.50%) and **147,365,888 bytes**
(+2.91%); both were below the respective pre-return peaks of **149,053,440**
and **150,142,976 bytes**, and comfortably passed the 20% bounded-recovery
gate.  In this particular shared-Windows-host measurement, neither final
sample immediately fell below its directly preceding `2560x1440` sample
(+1.67% / +1.76%), and working set showed the same allocator-capacity
retention.  This is reported as `declined_after_largest=false`, not presented
as an immediate decommit: the completed large image and its inference
temporaries have already been destroyed before measurement.  The durable
per-resolution report is
`build/memory_ladder_medium_cpu_tile4_20260813.json`.

### 2026-08-13: tiny resolution-ladder recovery rerun

The same native small-to-large-to-small memory gate was also run against the
tiny PP-OCRv6 model with 16 CPU workers and `PPOCR_APPROX_GELU=1`.  It uses
the deterministic sequence `16x16 -> 64x64 -> 160x120 -> 320x240 ->
640x480 -> 1280x720 -> 1920x1080 -> 2560x1440 -> 64x64`, sampling only after
each decoded image and its inference temporaries have gone out of scope.

Both cycles passed the 20% bounded-recovery gate from a **9,195,520-byte**
warmed private-byte baseline.  Immediately after `2560x1440`, the final
`64x64` sample fell from **9,940,992** to **9,232,384 bytes** (**-7.13%**) in
cycle one, and from **10,711,040** to **9,330,688 bytes** (**-12.89%**) in
cycle two.  Working set declined in both cycles as well.  This directly
confirms recovery after the largest image on the current host; the durable
per-resolution report is `build/memory_ladder_tiny_cpu_latest.json`.

### 2026-08-13: Vulkan terminal-GELU chain and immutable-chain cache

The Vulkan MobileNet `depthwise -> pointwise` chain now has an optional mode
that applies the existing opt-in AVX2/AVX-512 GELU approximation in its final
GPU writeback.  It avoids the former GPU readback followed by a full CPU GELU
pass for the medium-only graph fusion.  This path is selected only when
`PPOCR_APPROX_GELU=1` and the CPU dispatcher would make that same approximate
choice for the same logical tensor size; exact ONNX Erf-GELU, ARM/NEON, and
small tensors keep their established CPU behavior.  The chain also reuses its
immutable depthwise weights, bias, and packed pointwise parameters across
consecutive submissions, while invalidating that cache whenever a shared
Vulkan scratch binding is overwritten.

The AMD Radeon 780M Vulkan smoke validates both regular and GELU-fused chain
outputs against the shared CPU kernels.  Hybrid admission still measures the
complete upload, dispatch, fence, and readback boundary and only chooses GPU
when it is no slower; on this host the representative depthwise-pointwise
chain remained CPU-preferred (**0.601 ms GPU / 0.183 ms CPU**), while
affine+Swish and depthwise+Swish primitives were admitted in the same smoke.
This is intentional: the change makes a qualifying GPU segment cheaper
without relaxing the no-slower policy.

With 16 workers and `PPOCR_APPROX_GELU=1`, decoded-text parity versus ONNX
Runtime passed all four diverse inputs for each official model (**12/12**):
`16x16`, English, low contrast, and `3000x80` ultra-wide text.  The paired
eight-run `en.ppm` CPU/hybrid snapshots were tiny **32.042 / 34.855 ms**,
small **63.781 / 70.924 ms**, and medium **191.886 / 219.099 ms**.  Thus this
UMA run correctly retains CPU for the complete models; it is not presented as
a blanket GPU speed claim.  Reports are
`build/accuracy_{tiny,small,medium}_vulkan_gelu_fusion_20260813.json` and
`build/bench_{tiny,small,medium}_cpu_hybrid_gelu_fusion_20260813.json`.

The two-cycle medium `16x16 -> ... -> 2560x1440 -> 64x64` CPU memory ladder
also passed from a **147,148,800-byte** warm baseline.  Private bytes fell
immediately after the largest image in both cycles (**-2.01%** and **-1.39%**)
and working set fell as well; the durable report is
`build/memory_ladder_medium_cpu_vulkan_gelu_fusion_20260813.json`.

### 2026-08-13: refreshed ascending-resolution memory measurement

A fresh two-cycle medium CPU run (16 kernel workers,
`PPOCR_APPROX_GELU=1`) reconfirmed the deterministic
`16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 -> 1920x1080
-> 2560x1440 -> 64x64` sequence.  The checker samples process private bytes
and Windows working set only after each source image and its inference result
have left scope.  It passed the 20% bounded-recovery gate from a
**143,245,312-byte** warmed private baseline.

In cycle 1 the final `64x64` sample was **146,501,632 bytes** (+2.27% from
baseline, -0.63% from the pre-return private-byte peak); in cycle 2 it was
**143,319,040 bytes** (+0.05%, -4.66%).  The returned small image was below
the pre-return private-byte and working-set peaks in both cycles.  It was
**1.89%** above the immediately preceding `2560x1440` private-byte sample in
cycle 1, but **2.04%** below it in cycle 2; the working set follows the same
pattern.  Therefore the report truthfully records
`declined_after_largest=false` for cycle 1 and `true` for cycle 2 instead of
claiming unconditional immediate decommit.  This distinguishes bounded
live-memory recovery from Windows allocator capacity retention.  Full
per-resolution evidence is in
`build/memory_ladder_medium_cpu_latest_20260813.json`.

### 2026-08-13: SE-gate fusion investigation

The detector's largest squeeze-excitation block (`ReduceMean -> 1x1 -> ReLU
-> 1x1 -> HardSigmoid -> channel Mul`) was profiled as a large-page memory and
traffic candidate.  A strict graph fusion was added behind
`PPOCR_ENABLE_SE_GATE_FUSION=1`: it retains the dying NCHW source allocation,
forms only compact `[N,C,1,1]` gate storage, then scales the source in place.
It preserves the original per-channel FP32 reduction and gate order, and the
tiny/small/medium text regression passed **12/12** cases with it enabled.

It is intentionally **not** a production default yet.  On the current
16-worker AVX-512 host, the 30-run `en.ppm` A/B was favourable for tiny
(**30.926 -> 29.772 ms**, 1.039x), but regressed small (**61.034 -> 63.710
ms**) and was neutral for medium (**205.271 -> 205.028 ms**).  A three-run
medium `2560x1440` page A/B was also within noise but not a reliable win
(**2924.075 -> 2952.852 ms**).  Retaining this as opt-in prevents a
memory-oriented fusion from silently violating the all-three-model latency
requirement; the exact gate pattern remains available for a future
device-resident Vulkan chain.

After this investigation, a fresh medium two-cycle resolution ladder again
passed the 20% bounded-recovery gate.  Its warmed private-byte baseline was
**143,261,696 bytes**.  The final `64x64` sample fell below the immediately
preceding `2560x1440` sample in cycle 1 (**-1.70%**) and remained within the
gate in cycle 2 (**+2.13%**); working-set results and every resolution are in
`build/memory_ladder_medium_cpu_post_se_experiment_20260813.json`.

### 2026-08-13: AVX-512 stride-two NCHW batch tile

The AVX-512 `3x3, stride=2` NCHW batch scheduler now assigns one task to a
four-output-channel tile instead of invoking the kernel one channel at a
time.  Its SIMD body therefore shares every gathered/packed source vector
across four filters, exactly as the single-image dispatcher already did.
Output-channel ownership remains disjoint and each output retains the same
ascending FP32 accumulation order.  The optimization is dynamically entered
only after the existing AVX-512 runtime feature check; AVX2, scalar, and ARM
NEON paths are unchanged.  Set
`PPOCR_DISABLE_AVX512_STRIDE2_BATCH_TILE4=1` for a deployment A/B fallback.

The low-level kernel smoke and Vulkan smoke passed.  Decoded-text comparison
against ONNX Runtime passed all four varied inputs (`16x16`, English,
low-contrast, `3000x80`) for tiny, small, and medium (**12/12**); reports are
`build/accuracy_{tiny,small,medium}_stride2_batch_tile4_20260813.json`.

On four equal English pages, real detector and recognizer NCHW batches
(`det_batch=4`, `rec_batch=4`, exact width, two warm-ups, twelve timed runs)
remained serial-result-identical and measured these complete
serial-to-batch speedups on the 16-worker AVX-512 host: **1.573x** tiny
(184.036 / 116.981 ms), **1.379x** small (407.445 / 295.523 ms), and
**1.279x** medium (1105.502 / 864.359 ms).  This exercises the new stride-two
batch route directly, rather than inferring a batch benefit from a single
page.  Exact sample records are
`build/batch_{tiny,small,medium}_stride2_batch_tile4_20260813.txt`.

The same refreshed eight-run single-English CPU/Hybrid check measured
CPU **33.938 / 71.164 / 206.282 ms** and Hybrid
**36.744 / 74.706 / 292.331 ms** for tiny/small/medium.  On this UMA Radeon
780M host, Hybrid remains slower end-to-end, while individual admitted
Vulkan affine+Swish and depthwise+Swish segments still obey the
transfer-inclusive no-slower policy.  The result is not presented as a
whole-model GPU speed claim.

### 2026-08-13: heterogeneous-size batch regression

`bench/heterogeneous_batch_regression.py` adds a repeatable multi-size page
batch gate around `ppocr_batch_smoke`.  Its PPM corpus deliberately combines
`16x16`, ordinary two-line text, low-contrast text, `3000x80` wide text,
`80x3000` portrait text, a dense `1920x1080` UI page, and a `2560x1440`
screenshot.  It executes both exact-width (`PPOCR_REC_WIDTH_BUCKET=1`) and
a candidate padded-width grouping, while `ppocr_batch_smoke` verifies every
batch result is identical to serial inference.  The JSON records both timing
samples rather than promoting a padding policy from a narrow equal-page test.

On the current 16-worker AVX-512 CPU with approximate GELU, all three models
passed serial-identity on this seven-image / 186--188-result corpus.  The
exact-width batch means were **473.748 ms** (tiny), **1771.887 ms** (small),
and **8437.291 ms** (medium), with serial-to-batch speedups of **1.258x**,
**1.156x**, and **1.038x**, respectively.  A 16-pixel candidate bucket
measured **487.901 ms** (+3.0%), **1796.819 ms** (+1.4%), and **8945.280 ms**
(+6.0%).  Exact-width therefore remains the lower-memory CPU default; the
candidate remains an exposed deployment knob for workloads where reduced
shape fragmentation pays back its bounded right-padding.  Reports are in
`build/heterogeneous_batch_{tiny,small,medium}_bucket16_20260813.json`.

### 2026-08-13: batch-invariant GELU policy

The selective GELU dispatch now receives the logical per-sample element count
from every generic and fused graph route.  Consequently, packing four equal
pages/crops into an NCHW batch cannot change a short tensor from exact Erf to
the opt-in approximate formula merely because the batch has four times as many
elements.  This closes a batch-correctness edge case while keeping the medium
large-map AVX-512/AVX2 acceleration.  The low-level GELU smoke passed in both
exact and approximate configurations, and all three models again passed the
four-image ONNX Runtime decoded-text gate (**12/12**).

An N=4 CPU public-batch smoke with `det_batch_size=4`, `rec_batch_size=4` and
four-way preprocessing remained serial-result-identical for every model. On
this shared AVX-512 host it measured tiny **1.346x**, small **1.090x**, and
medium **1.055x** serial-to-batch speedup.  A new paired medium CPU A/B (two
warm-ups, eight runs) measured **211.217 ms** with the corrected SIMD route
and **219.778 ms** with `PPOCR_DISABLE_APPROX_GELU_SIMD=1` (**3.9%** lower
mean latency). Hybrid measured **314.123 ms** in the same short sample; the
strict transfer-inclusive Vulkan admission therefore remains necessary and
continues to retain CPU for transfer-bound operators.

### 2026-08-13: Vulkan batch-admission parity correction

The per-shape Vulkan admission probes for pointwise Conv and residual
pointwise Conv now benchmark the same batched CPU SIMD kernels used by live
`[N,C,H,W]` OCR execution.  Previously these two probes used a serial loop of
single-image kernels even though the executor flattens batch/channel tiles;
that could make a GPU segment appear no slower than a weaker CPU reference.
The correction does not change model arithmetic or force a GPU path. It makes
the hybrid policy stricter and accurate: Vulkan is selected only when its full
upload, dispatch, synchronization, readback, and validation boundary is no
slower than the real batched CPU alternative.

`ppocr_vulkan_smoke` and the full SIMD kernel smoke passed on the local AMD
Radeon 780M / AVX-512 host.  The current primitive measurements correctly
retain CPU for pointwise Conv (`0.22340 ms` GPU versus `0.02218 ms` CPU) and
residual pointwise Conv+ReLU (`0.24984 ms` versus `0.03832 ms`), while keeping
GPU for the validated depthwise+Swish sample (`0.41966 ms` versus `0.42462
ms`).  This is explicit evidence of the requested no-slower hybrid policy,
not a claim of universal GPU speedup or GPU-only availability.

### 2026-08-13: current three-model CPU/hybrid snapshot

With `PPOCR_BENCH_THREADS=16` and `PPOCR_APPROX_GELU=1`, a refreshed CPU
measurement on the two-line English PPM used two warm-ups and twelve timed
runs.  Tiny measured **33.025 ms** (30.280 FPS), small **70.530 ms** (14.178
FPS), and medium **246.420 ms** (4.058 FPS).  A controlled medium worker scan
on the same shared host measured 8 / 12 / 16 workers at **366.673 / 282.361 /
246.420 ms**, respectively; this validates the current 16-worker ceiling for
the broad medium graph without claiming that all deployments share that curve.

Hybrid mode with the same model/input and two warm-ups/eight runs measured
tiny **37.400 ms**, small **75.953 ms**, and medium **235.987 ms**.  Medium
therefore saw a **4.2%** lower mean latency on this local hybrid run, while
tiny/small correctly remain CPU-preferred at the segment level when transfers
do not pay back.  A paired medium ONNX Runtime 1.28 CPUExecutionProvider run
(matching 16-thread budget, two warm-ups/eight runs) measured **399.188 ms**;
the native medium CPU snapshot is **38.3%** lower latency and hybrid is
**40.9%** lower on this host.  These are shared-host measurements rather than
portable guarantees; model accuracy and the hybrid admission rule remain
independent of benchmark mode.

`bench/stress_regression.py` now resolves a default `.png` case to a sibling
`.ppm` case when necessary. The checked-in native regression corpus can
therefore run directly without duplicating the large source images or adding
an image decoder to the C++ executable. The tiny-model CPU stress gate passed
all **11/11** cases: dense UI, blank, single word, ultra-wide/tall strips,
tiny, low-contrast, rotated, dense blobs, and two screenshot sizes.

### 2026-08-13: Vulkan GELU-chain smoke repair and refreshed checks

The Vulkan smoke's depthwise-to-pointwise mode 30 test now calculates the
same explicit Pad鑼?tanh GELU approximation as the shader. Previously the test
accidentally compared this opt-in approximation with exact `kernels::Gelu()`
when `PPOCR_APPROX_GELU` was unset, producing a false failure even though the
shader's mode was correct. This is a test-only correction: it does not change
the default exact-GELU execution policy or GPU admission rule. The rebuilt
AVX-512 kernel smoke and Radeon 780M Vulkan primitive suite passed, including
the device-resident depthwise閳姫ointwise閳墱ELU chain.

The refreshed CPU-versus-ORT `en.ppm` comparison (16 kernel workers, opt-in
approximate GELU, 2 warm-ups/20 timed runs) measured native tiny **40.480 ms**
versus ORT **122.008 ms** (**66.8% lower latency**), small **79.199 ms** versus
**183.173 ms** (**56.8% lower**), and medium **242.245 ms** versus **291.551
ms** (**16.9% lower**).  These are local shared-host measurements, not a
portable guarantee. Decoded-text parity against ORT also passed four diverse
inputs for each model (English, mixed text, single-line text, and low-contrast
multi-line text): **12/12**.

Hybrid retains the strict full-boundary admission rule and therefore only
uses a Vulkan segment when its measured H2D閳妿ispatch閳姁ence閳墬2H result is no
slower than the matching CPU SIMD work. A seven-image varied-size hybrid batch
(16x16, English, low contrast, ultra-wide, ultra-tall, 1920x1080 UI, and
2560x1440 screenshot) stayed serial-result-identical and took **850.547 ms**
batched versus **1049.532 ms** serial (**1.234x**) with four graph workers.
A two-cycle tiny Hybrid resolution ladder also passed every bounded memory
recovery limit; its final `64x64` samples stayed below the warmed
119,021,568-byte private baseline. `gpu_only` remains deliberately rejected:
the complete OCR graph and DB post-processing are not yet device-resident, so
silently using a partial GPU path would violate the public backend contract.

### 2026-08-13: refreshed ascending-resolution memory recovery check

The repository includes a reproducible ascending-resolution PPM ladder and a
native process-memory checker: `bench/generate_memory_ladder.py` creates
`16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 -> 1920x1080
-> 2560x1440 -> 64x64`; `ppocr_mem_soak` loads, infers, and destroys each
image before sampling Windows private bytes and working set.  The final
`64x64` sample is therefore a direct observation after the largest image,
rather than a retained source-image allocation.  The checker requires the
returned small-image private bytes to stay within 20% of the warmed baseline,
the preceding `1920x1080` sample, and the immediately preceding largest-page
sample; it additionally records whether private bytes and working set actually
declined after `2560x1440`.

A fresh two-cycle CPU run with `PPOCR_BENCH_THREADS=16` and
`PPOCR_APPROX_GELU=1` passed for both available dictionary/model pairs. Tiny
returned from **10,756,096 B** at `2560x1440` to **10,199,040 B** (-5.18%) in
cycle 1 and from **9,953,280 B** to **9,420,800 B** (-5.35%) in cycle 2;
private memory therefore declined directly after the large page in both
cycles. Medium returned from **146,243,584 B** to **143,228,928 B** (-2.06%)
in cycle 1; cycle 2's returned sample was **146,063,360 B**, 1.58% above that
cycle's immediate `2560x1440` sample but still below its pre-return peak and
0.62% below the warmed baseline. Both models met every bounded-recovery gate.
The complete per-resolution private-byte and working-set records are retained
in `build/memory_ladder_tiny_cpu_resolution_recheck_20260813.json` and
`build/memory_ladder_medium_cpu_resolution_recheck_20260813.json`. These are
host-specific allocator observations, not a guarantee that every platform will
immediately decommit heap capacity after a large image.

### 2026-08-13: SE workspace reuse and refreshed medium memory ladder

`SqueezeExcitationGateInplace` now stores one `[N,C]` gate/mean buffer and a
single reusable `[reduced-C]` hidden row.  The former implementation retained
two `[N,C]` temporary buffers plus `[N,reduced-C]`; the helper's temporary
workspace is consequently reduced from `2*N*C + N*reduced-C` to
`N*C + reduced-C` floats, without changing its FP32 accumulation order or
the completed gate values.  This is a local SE-workspace reduction, not a
claim that total process memory drops by the same amount.

The rebuilt SIMD and Vulkan smoke suites passed, and decoded-text parity with
ONNX Runtime remained **4/4 for each of tiny, small, and medium** across
English, mixed, single-line, and low-contrast inputs.  The refreshed tiny and
small reports are
`build/accuracy_{tiny,small}_cpu_se_workspace_20260813.json`; medium was split
into `build/accuracy_medium_cpu_se_workspace_20260813_part{1,2}.json` to keep
each check bounded.  A fresh two-cycle CPU ascending-resolution run
(`16x16 -> ... -> 2560x1440 -> 64x64`, 16 workers, approximate GELU) also
passed every 20% bounded-recovery gate.  Its warmed private baseline was
**147,709,952 B**; the final small image was **143,351,808 B** and
**143,364,096 B**, respectively.  Private bytes and working set both declined
directly after `2560x1440` in both cycles.  Per-resolution private and working
set samples are retained in
`build/memory_ladder_medium_cpu_se_workspace_20260813.json`.

### 2026-08-13: extended-resolution and aspect-ratio memory check

The reproducible ladder now also covers `3000x1600`, `3840x2160`, the
ultra-wide `4096x256`, and ultra-tall `256x4096` before returning to `64x64`.
This exercises both pixel-count growth and detector resize branches that a
landscape-only corpus misses.  On the local 16-worker CPU, both tiny and
medium passed two cycles of this extended sequence.  For medium the
143,294,464-byte warmed baseline returned to **143,347,712 B** (+0.04%) and
**143,392,768 B** (+0.07%); both samples were below the immediately preceding
`3840x2160` page and their cycle high-water marks, and working set declined as
well.  Tiny returned to **10,211,328 B** (+1.47%) and **9,596,928 B** (-4.64%)
from its 10,063,872-byte baseline; its private bytes declined after the 4K
page in both cycles.  Its first-cycle working set did not immediately decline
from the preceding 4K sample, but remained below the cycle high-water mark;
the second cycle did decline.  Full durable resolution-by-resolution reports are
`build/memory_ladder_{tiny,medium}_cpu_extended_aspect_20260813.json`.

### 2026-08-13: staged post-4K memory recovery check

The memory ladder now continues after `3840x2160` with `1920x1080`,
`4096x256`, and `256x4096` before its final `64x64` sample. This distinguishes
recovery after a large area from recovery after the detector's two extreme
aspect-ratio resize paths. A fresh medium CPU pass (16 workers, approximate
GELU) warmed at **144,003,072 B** private bytes, peaked at
**149,565,440 B**, and returned to **144,084,992 B** after the full tail
(+0.06% over the warm baseline; -3.31% versus the 4K sample). Its working set
also returned from the 4K **152,662,016 B** to **147,812,352 B**. The durable
per-resolution report is
`build/memory_ladder_medium_cpu_staged_recovery_20260813.json`. These are
local allocator observations, not a cross-platform decommit guarantee.

### 2026-08-13: selective default SE graph fusion

SE gate fusion is now enabled by default only for detector blocks with at
least 512 input channels, which selects the three high-cost medium blocks and
keeps tiny/small on their lower-overhead established paths.  The full-shape
experiment remains available through `PPOCR_ENABLE_SE_GATE_FUSION=1`; disable
all default fusion with `PPOCR_DISABLE_SE_GATE_FUSION=1`, or tune the boundary
with `PPOCR_SE_GATE_FUSION_MIN_CHANNELS`.  This is a genuine graph/operator
fusion: it replaces `ReduceMean -> Conv -> Relu -> Conv -> HardSigmoid -> Mul`
with one in-place gate operation, avoiding the broadcast-map destination and
the intermediate graph dispatches.

On the same shared 16-worker AVX-512 host, a 2-warm-up/20-run English-page
CPU snapshot measured **30.982 ms** (tiny), **64.365 ms** (small), and
**195.898 ms** (medium), respectively.  The medium run before this selective
default measured 210.198 ms in a separate 12-run sample; this comparison is
directional because the host is shared, rather than a portable performance
claim.  The existing 20-run CPU-versus-ORT measurements remain the published
cross-engine evidence.  Kernel and Vulkan smoke both passed, and medium
retained ONNX Runtime decoded-text parity on all four diverse images; reports
are `build/bench_{tiny,small,medium}_cpu_selective_sefusion_20260813.json`
and `build/accuracy_medium_cpu_selective_sefusion_20260813_part{1,2}.json`.

### 2026-08-13: folded Conv channel-bias execution path

The graph loader already recognizes the recognizer pattern `Conv ->
Add([1,C,1,1]) -> Identity` and precomputes the resulting per-output-channel
bias. Its executor now passes that folded bias directly to the convolution,
removing the accidental second `AddChannelBias` traversal that had remained
after load-time folding. This keeps the original FP32 order閳ユ攦onvolutional
reduction followed by the same accumulator bias閳ユ攣nd avoids a full NCHW
read/write for each matched node. Set `PPOCR_DISABLE_FOLDED_CHANNEL_BIAS=1`
for a runtime A/B comparison.

All three model sizes retained decoded-text parity with ONNX Runtime on the
four-image English/mixed/single-line/low-contrast corpus (12/12 cases):
`build/accuracy_{tiny,small,medium}_cpu_folded_channel_bias_20260813.json`.
Kernel and Vulkan smoke both passed. On this shared 16-worker AVX-512 host,
a 2-warm-up/16-run CPU snapshot measured **31.574 ms** (tiny), **67.060 ms**
(small), and **201.835 ms** (medium); the matching files are
`build/bench_{tiny,small,medium}_cpu_folded_channel_bias_20260813.json`.
Separate 40-run A/B samples showed a medium reduction from **211.568 ms** to
**201.835 ms** (4.6%), while tiny/small were within normal shared-host noise;
this is therefore retained as a medium-oriented optimization rather than a
claim of uniform speedup on every model or system.

The staged medium CPU memory test also passed after the change:
**143,548,416 B** warmed private bytes, **151,822,336 B** at 4K, and
**143,601,664 B** at the final `64x64` image (+0.04% versus warm; -5.41%
versus 4K). Working set fell from **153,632,768 B** at 4K to
**147,640,320 B**. The durable report is
`build/memory_ladder_medium_cpu_folded_channel_bias_20260813.json`.

### 2026-08-13: refreshed full-resolution memory recovery measurement

The reproducible native memory ladder was rerun after rebuilding the current
CPU/Vulkan sources.  It processes one decoded RGB image at a time in this
order: `16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 ->
4096x256 -> 256x4096 -> 1920x1080 -> 2560x1440 -> 3000x1600 -> 3840x2160 ->
1920x1080 -> 4096x256 -> 256x4096 -> 64x64`.  Thus the last small image is
sampled only after the largest-area page and both detector extreme-aspect-ratio
paths have completed and their input images have been destroyed.

With medium PP-OCRv6, CPU backend, 16 kernel workers, and approximate GELU,
the warmed private baseline was **144,879,616 B**.  The `3840x2160` sample was
**151,138,304 B** and the pre-return high-water mark was **152,875,008 B**;
the final `64x64` sample fell to **143,867,904 B** (-0.70% from warm, -4.81%
from the 4K sample, and -5.89% from the cycle high-water mark).  Its Windows
working set likewise fell from **153,780,224 B** at 4K (cycle peak
**156,557,312 B**) to **147,533,824 B** at the final small image.  The
bounded-recovery gate passed and both explicit decline flags are true.  This
is process-level evidence for this host and build, not a promise that every
allocator or Vulkan driver will immediately decommit reusable pages.  The
per-resolution report is
`build/memory_ladder_medium_cpu_post_vulkan_tile_rebuild_20260813.json`.

### 2026-08-13: current small-to-large-to-small memory regression

The current CPU build was also measured with a fresh, reproducible resolution
ladder: `16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 ->
4096x256 -> 256x4096 -> 1920x1080 -> 2560x1440 -> 3000x1600 -> 3840x2160`,
followed by `1920x1080 -> 4096x256 -> 256x4096 -> 64x64`.  Each sample loads,
recognizes, and destroys its input image before Windows private bytes and
working set are sampled.

On medium PP-OCRv6 with CPU backend, 16 kernel workers, and approximate GELU,
the warmed private baseline was **143,552,512 B**.  Private bytes reached
**149,356,544 B** at `3840x2160` and fell to **148,066,304 B** at the final
`64x64` sample: **-0.86%** from the 4K sample and **+3.14%** from warm.  The
working set likewise fell from **153,243,648 B** to **151,785,472 B**.  The
bounded-recovery gate passed; both private-byte and working-set decline flags
are true.  This verifies a post-large-image decrease on this host, while the
small residual capacity is expected allocator reuse rather than a leak claim.
The full per-resolution JSON record is
`build/memory_ladder_medium_cpu_resolution_ladder_current_20260813.json`.

### 2026-08-13: dense-page scheduler and model-aware stress recheck

The default CPU crop-graph concurrency is now **8** rather than 12.  This
matches the default SIMD kernel worker budget, avoids cache/bandwidth
contention between outer crop batches and inner AVX-512/AVX2 kernels, and
keeps the public `PPOCR_REC_PARALLELISM` override for host-specific tuning.
On the local AVX-512 CPU (16 configured kernel workers, approximate GELU),
the normal English page measured **33.548 / 70.323 / 210.214 ms** for
tiny/small/medium (2 warm-ups, 8 runs).  Medium was **6.6%** lower than the
comparable 12-way outer-crop configuration (225.056 ms); this is a local
configuration measurement, not a portable performance guarantee.

All three model sizes preserved decoded-text parity against ONNX Runtime over
four varied inputs each (**12/12**): English, mixed text, a single-line crop,
and low-contrast text.  The reports are
`build/accuracy_{tiny,small,medium}_cpu_default8_20260813.json`.
`bench/stress_regression.py` now accepts `--model tiny|small|medium` and has
explicit medium-only box-count bounds for the synthetic blank and dense-blob
cases. This preserves the original tiny/small limits while avoiding a false
failure when the larger detector stably reports its extra valid components.
The 11-image medium stress corpus passed **11/11** with `--model medium`;
the record is `build/stress_medium_cpu_default8_model_bounds_20260813.json`.

### 2026-08-13: dense-page activation bound and refreshed memory check

The default recognizer NCHW batch is now **1** crop (the public
`PPOCR_REC_BATCH_SIZE` override still enables larger batches for measured
multi-page workloads).  A current dense `1920x1080` UI sweep showed that
larger concurrent crop tensors increase medium-model activation pressure and
lose latency on this host.  With CPU, 16 SIMD workers, approximate GELU, and
the eight-way outer scheduler, the default measured **255.141 ms** (tiny,
2 warm-ups/6 runs), **939.888 ms** (small, 2/4), and **3884.903 ms** (medium,
2/4).  Against the matching CPU ONNX Runtime pipeline on the same dense page,
a separate 2-warm-up/4-run sample was **46.0%**, **18.7%**, and **5.9%** lower
latency for tiny/small/medium respectively.  The latter remains a
machine-specific comparison; the dense image and detected-box mix are part of
the benchmark input.  Reports: `build/dense_cpu_batch1_vs_ort_20260813.json`
and `build/dense_batch_size_sweep_cpu_20260813.json`.

The change retained three-model decoded-text parity on four varied inputs
(**12/12**; `build/accuracy_{tiny,small,medium}_cpu_default_batch1_20260813.json`).
A fresh medium CPU resolution ladder also passed.  It warmed at
**143,974,400 B**, reached **148,078,592 B** at `3840x2160`, and returned to
**144,060,416 B** at the final `64x64` image: **-2.71%** versus the 4K sample
and **+0.06%** versus warm.  Windows working set declined from
**151,060,480 B** to **147,656,704 B**; both recovery flags are true.  The
durable per-resolution record is
`build/memory_ladder_medium_cpu_default_batch1_20260813.json`.

### 2026-08-13: recognizer executor allocation reuse

`RunCtcTop1`, the production recognizer route, now reuses a per-worker
thread-local small input-pointer list for its fused and generic node dispatch.
This removes recurring short-lived `std::vector<const Tensor*>` allocations
for every recognizer node and crop while retaining the existing tensor
lifetime plan, exact operator calls, and safe automatic growth for a future
larger input arity. It is deliberately a low-risk allocation/allocator
pressure reduction rather than a claim of an isolated compute-kernel speedup.

After the change, decoded text still matched ONNX Runtime on the four varied
images for tiny, small, and medium (**12/12**), and kernel/Vulkan smoke passed.
A new medium CPU resolution ladder also passed with the full ascending and
recovery sequence. Its private bytes fell from **150,818,816 B** at
`3840x2160` to **143,855,616 B** at the final `64x64` (**-4.62%**); working
set fell from **154,529,792 B** to **147,832,832 B**. The final sample was
2.60% below the warmed baseline and all bounded-recovery/decline checks were
true. See `build/memory_ladder_medium_cpu_executor_scratch_20260813.json`.

The immediate dense-page sanity sample (CPU, 16 kernel workers, approximate
GELU, default recognizer batch) was **255.959 ms / 860.055 ms / 3859.129 ms**
for tiny/small/medium (one warm-up, three runs). Because shared-host timing
varies materially, it is retained only as a post-change sanity record rather
than compared as a claimed percentage gain. The three JSON records are
`build/executor_scratch_bench_{tiny,small,medium}_cpu_20260813.json`.

Hybrid remains guarded by per-shape full-boundary admission (`H2D / dispatch
/ fence / D2H`); it uses Vulkan only when that complete operation is no
slower than the matching CPU SIMD batch kernel. On the local AMD Radeon 780M
snapshot, Hybrid measured 33.469/68.070/264.781 ms for tiny/small/medium,
respectively, so the end-to-end route is still not a GPU speedup on this UMA
device. `gpu_only` remains unavailable until the complete OCR graph and DB
post-processing can remain device-resident.

### 2026-08-13: compact CTC last-use counters

The recognizer's `RunCtcTop1` no longer clones a string-keyed
`unordered_map` of tensor-use counts for every crop. Model loading now assigns
each dynamic activation one compact counter slot; each worker resets a
thread-local `uint16_t` counter vector and continues to resolve only the
fixed names needed by the existing lifetime planner. This removes repeated
map-node allocation/copying from the multi-crop hot path while preserving the
same final-use, in-place, and immediate-release decisions.

The native kernel/Vulkan smoke suites passed, as did decoded-text parity
against ONNX Runtime for all three model sizes over English, mixed text,
single-line, and low-contrast inputs (**12/12**). A heterogeneous seven-image
tiny Hybrid batch (`16x16`, normal text, low contrast, ultra-wide,
ultra-tall, dense 1080p, and 1440p screenshot) remained serial-identical for
all 188 results. The one-run steady-state observation was **1456.274 ms**
batched versus **1485.160 ms** serial; it is not promoted to a general GPU
latency claim.

The accompanying medium CPU full-resolution ladder passed its 20% bounded
recovery gate. The final small image used **148,414,464 B** private memory
(+0.25% versus its warmed baseline and 0.65% below the cycle peak). This
particular Windows allocator run did not immediately decommit relative to the
`3840x2160` checkpoint, so both direct-decline flags are intentionally
recorded as false rather than presented as a memory-reduction claim. Full
per-resolution evidence is
`build/memory_ladder_medium_cpu_ctc_use_slots_20260813.json`.

The dense timing samples were materially host-noisy (tiny **361.536 ms**,
small **1858.371 ms**, medium **4585.517 ms** means) and therefore serve only
as post-change sanity records, not percentage-improvement evidence:
`build/ctc_use_slots_bench_{tiny,small,medium}_cpu_20260813.json`.

### 2026-08-13: explicit post-4K resolution-memory recovery sample

The native memory regression now distinguishes the **first** image after the
`3840x2160` sample (`1920x1080`) from the final `64x64` check.  Its JSON
includes private-byte and working-set decline flags for that immediate 4K
transition, in addition to the existing bounded final-small-image gates. This
avoids accidentally treating a later aspect-ratio sample as proof of direct
large-image recovery.

On this Windows host, a fresh one-cycle medium CPU run over the deterministic
ascending ladder (`16x16` through `3840x2160`) passed all 20% bounded-recovery
limits. Private usage fell from **148,787,200 B** at `3840x2160` to
**148,303,872 B** after the immediately following `1920x1080` input
(**-0.32%**); working set fell from **152,776,704 B** to **151,871,488 B**.
After the staged wide/tall recovery tail, the final `64x64` sample was
**143,667,200 B** private (**-3.08%** versus the warmed baseline). This is a
host/run-specific observation閳ユ摱indows may retain heap capacity on a later run;
the report preserves both direct-decline booleans and all per-resolution
samples. Reproduce it with `bench/generate_memory_ladder.py` and
`bench/memory_ladder_regression.py`; recorded evidence:
`build/memory_ladder_medium_cpu_immediate_4k_recovery_20260813.json`.

### 2026-08-13: current full-resolution memory-recovery check

The deterministic native memory ladder was run again with the current release
build and medium PP-OCRv6 on the CPU backend.  It processes one image at a
time, releases that image before sampling, and covers the complete sequence:
`16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 -> 4096x256 ->
256x4096 -> 1920x1080 -> 2560x1440 -> 3000x1600 -> 3840x2160 -> 1920x1080
-> 4096x256 -> 256x4096 -> 64x64`.

On this Windows host, private bytes were **148,770,816 B** at `3840x2160` and
fell immediately to **147,767,296 B** after the next `1920x1080` image
(**-0.67%**).  The working set also fell from **151,744,512 B** to
**150,671,360 B**.  After the full recovery tail, the final `64x64` sample was
**147,222,528 B** private and **151,085,056 B** working set: below the 4K
sample in both measures.  All bounded-recovery gates passed.  This is a
host-specific allocator observation, not a promise that every OS allocator
will immediately decommit reusable heap capacity.  The complete per-resolution
record is `build/memory_ladder_medium_cpu_current_20260813.json`.

### 2026-08-13: generic SIMD Conv+ReLU correctness and memory recheck

The generic AVX-512/AVX2 spatial-convolution fallback now applies its requested
ReLU after the vector convolution writeback. Dedicated fused 2x2/3x3 paths
already performed that activation, but the generic 5x5/7x7 and asymmetric
SIMD fallback did not. This is a correctness repair, not a claimed latency
win. The rebuilt kernel and Vulkan smoke tests pass, and decoded-text parity
against ONNX Runtime passes **4/4** varied inputs for every PP-OCRv6 tier
(tiny, small, medium): English, mixed text, a short line, and noisy
low-contrast text. Reports are
`build/accuracy_{tiny,small,medium}_relu_fix_20260813.json`.

The medium CPU heterogeneous six-image batch smoke (`16x16`, normal English,
mixed text, wide/tall strips, low contrast) was result-identical and measured
**1653.548 ms serial** versus **1414.858 ms batched** (**1.169x** throughput).
A fresh full-resolution CPU memory ladder also passed: its `3840x2160` private
sample was **149,692,416 B**, the immediately following `1920x1080` sample
was **145,428,480 B** (**-2.85%**), and the final `64x64` sample was
**147,177,472 B** (**-0.35%** versus warm baseline). The complete report is
`build/memory_ladder_medium_cpu_relu_fix_recheck_20260813.json`.

### 2026-08-13: refreshed diverse CPU versus ONNX Runtime measurement

Using the current release build, dynamic AVX-512 dispatch, CPU-only backend,
`PPOCR_BENCH_THREADS=16`, five warm-ups, and 30 timed iterations per case, a
new paired measurement confirms a material end-to-end latency advantage over
ONNX Runtime CPUExecutionProvider for all three official PP-OCRv6 tiers on two
different input shapes.  This is host-specific evidence, not a cross-hardware
guarantee; both engines ran sequentially in the same process environment.

| Input | Model | C++ mean | ORT mean | C++ latency reduction |
|---|---|---:|---:|---:|
| `en.ppm` | tiny | 32.653 ms | 125.618 ms | 74.01% |
| `en.ppm` | small | 68.304 ms | 180.432 ms | 62.14% |
| `en.ppm` | medium | 208.100 ms | 366.127 ms | 43.16% |
| `wide_strip_3000x80.ppm` | tiny | 30.776 ms | 65.817 ms | 53.24% |
| `wide_strip_3000x80.ppm` | small | 60.182 ms | 129.017 ms | 53.35% |
| `wide_strip_3000x80.ppm` | medium | 175.134 ms | 272.745 ms | 35.79% |

The full machine-readable measurement is
`build/bench_cpu_ort_diverse_refresh_20260813.json`. It complements rather
than replaces the larger noisy/dense stress cases: those cases exercise memory
and batch robustness and can be affected more by host scheduling/cache noise.

### 2026-08-13: current CPU-versus-ORT recheck

After the current SIMD/scheduler rebuild, the paired CPU-only comparison was
rerun with dynamic AVX-512 enabled, `PPOCR_BENCH_THREADS=16`, two warm-ups,
and ten timed iterations per case.  The compact English page and a 3000x80
ultra-wide strip cover normal and extreme-aspect detector resize paths.  All
three tiers remain faster than ONNX Runtime CPUExecutionProvider on this host;
the lower reduction on medium's short page reflects host timing variance and
is recorded rather than hidden.

| Input | Model | C++ mean | ORT mean | C++ latency reduction |
|---|---|---:|---:|---:|
| `en.ppm` | tiny | 33.256 ms | 121.671 ms | 72.67% |
| `en.ppm` | small | 81.696 ms | 180.725 ms | 54.80% |
| `en.ppm` | medium | 246.343 ms | 374.196 ms | 34.17% |
| `wide_strip_3000x80.ppm` | tiny | 24.297 ms | 59.489 ms | 59.16% |
| `wide_strip_3000x80.ppm` | small | 53.559 ms | 101.561 ms | 47.26% |
| `wide_strip_3000x80.ppm` | medium | 152.553 ms | 263.361 ms | 42.07% |

The reproducible JSON record is
`build/bench_cpu_ort_diverse_current_20260813.json`.  It is a local-machine
measurement, not a guarantee for every processor or ONNX Runtime build.

### 2026-08-13: blocked recognizer transpose

The recurrent PP-OCRv6 recognizer exchanges both `[N,T,C] <-> [N,C,T]` and
`[N,H,C,T] <-> [N,H,T,C]` layouts.  Their dedicated copy paths now transpose
`32x32` cache tiles instead of continually evicting a full source/destination
matrix row.  The operation is a pure element relocation, so it preserves
values bit-for-bit and applies to scalar, AVX2, AVX-512, and NEON builds.

On this AVX-512 host with `PPOCR_BENCH_THREADS=16`, three independent runs of
12 timed English-page inferences changed mean latency from **32.120 to
30.530 ms** (tiny, **4.95%**), **62.793 to 64.053 ms** (small, **-2.01%**),
and **192.745 to 190.614 ms** (medium, **1.11%**).  The small-model result is
within host scheduling noise and is recorded rather than promoted as a broad
claim; no model-specific switch is required.  Raw paired samples are
`build/bench_transpose_tile_{baseline,tiled}_20260813.json`.  Text parity
against ORT remained 3/3 across English, ultra-wide, and low-contrast inputs
for all tiers in `build/accuracy_transpose_tiled_core_20260813.json`.

The same current medium build passed the full small-to-4K-to-small memory
ladder: `3840x2160` was **149,356,544 B** private / **153,313,280 B** working
set, the immediate following 1080p sample fell to **148,262,912 B** /
**152,317,952 B**, and the final `64x64` sample was **148,324,352 B** /
**152,129,536 B**.  Full per-resolution evidence:
`build/memory_ladder_medium_cpu_transpose_tiled_20260813.json`.

### 2026-08-13: CPU/Hybrid batch recheck and GPU-only boundary

The public three-policy contract was rechecked on the local Radeon 780M:
Vulkan loader and compute are present, while `gpu_only` explicitly rejects
construction because PP-OCRv6 still needs device implementations for its
complete detector, recognizer, preprocessing, and DB/CTC postprocessing
graphs.  It does not silently execute on CPU.  Hybrid remains usable and
continues to select each Vulkan segment only after its complete synchronous
`H2D + dispatch + fence + D2H` path is correct and no slower than CPU.

With four same-shape English pages, `det_batch_size=4`, `rec_batch_size=4`,
one graph worker, and two timed iterations, the verified `RecognizeBatch()`
result-identical speedups over four serial calls were **1.589x / 1.256x /
1.140x** on CPU for tiny/small/medium and **1.598x / 1.323x / 1.225x** in
Hybrid.  This is real cross-page NCHW batching, not padding unrelated page
sizes.  The raw output is `build/bench_batch4_cpu_hybrid_20260813.json`.

### 2026-08-13: AVX-512 recognizer GEMM tile admission

The AVX-512 GEMM implementation retains its two-row and four-row kernels,
but the four-row tile is now opt-in (`PPOCR_ENABLE_AVX512_GEMM4ROW=1`) rather
than the default.  A paired `PPOCR_BENCH_THREADS=16` measurement showed that
the larger register footprint is beneficial only for some medium shapes and
regresses the shipped workload overall.  With the default two-row tile and
three warm-ups/15 runs, current means were **28.885 / 64.840 / 201.871 ms**
on the English page and **20.263 / 48.142 / 134.558 ms** on the 3000x80 strip
for tiny/small/medium. The control data is
`build/bench_gemm4row_{ab_current,off_repeated}_20260813.json` and the final
default record is `build/bench_gemm2row_default_20260813.json`.

Text parity remains **3/3** for each model on English, ultra-wide, and
low-contrast inputs against ONNX Runtime:
`build/accuracy_gemm2row_core_20260813.json`. This change keeps the exact
per-row K-order and applies only after dynamic AVX-512 capability checks;
AVX2, NEON, and scalar dispatch are unchanged.

### 2026-08-13: hot-path ISA configuration caching

The runtime ISA policy remains dynamic (CPUID/XCR0 is still checked before
entering AVX-512), but process-scoped environment switches for pointwise
tiles, 2x2 tiles, wide tiles, and the 2x2 SAME kernel are now cached once.
These predicates occur repeatedly in the detector and recognizer; eliminating
repeated environment lookups reduces dispatcher overhead without changing an
operator's arithmetic or the AVX2/NEON/scalar fallback. On the current
16-worker AVX-512 host, the final English-page smoke measured **30.462 ms**
tiny, **62.268 ms** small, and **190.709 ms** medium. Raw result:
`build/bench_dispatch_cache_current_20260813.json`.

All three tiers retained 3/3 ORT decoded-text parity on English, ultra-wide,
and low-contrast inputs (`build/accuracy_dispatch_cache_core_20260813.json`),
and both CPU kernel and Vulkan primitive smoke tests passed.

### 2026-08-13: detector BatchNorm activation reuse

The generic graph executor now mirrors the production CTC executor for a
final-use plain BatchNorm: it applies the alias-safe affine transform directly
to the dying input activation, then transfers the tensor name to the graph
output. This removes one same-sized detector feature-map allocation/copy at
eligible boundaries without changing the BatchNorm arithmetic or its dynamic
coefficient handling. It is guarded by the existing
`PPOCR_DISABLE_BATCHNORM_INPLACE=1` deployment A/B switch.

All three model tiers retained 3/3 decoded-text parity against ONNX Runtime on
English, ultra-wide, and low-contrast inputs
(`build/accuracy_bn_inplace_core_20260813.json`); CPU and Vulkan smoke also
passed. The refreshed medium full ladder passed with a 4K private/working-set
sample of **148,881,408 B / 152,858,624 B**, immediately declining at the
following 1080p image to **147,857,408 B**, then returning to
**143,728,640 B / 147,775,488 B** on the final 64x64 image. Complete records:
`build/memory_ladder_medium_cpu_bn_inplace_20260813.json` and
`build/bench_bn_inplace_current_20260813.json`.

### 2026-08-13: single-pass SIMD CTC ArgMax

AVX-512 and AVX2 CTC ArgMax no longer scan each vocabulary row twice. The
SIMD reduction now retains each lane's source index as it computes its local
maximum, then performs one scalar lane/tail reduction with the same strict
`>` first-maximum rule as the portable path. This removes a second full read
of each 6,906/18,710-class logits row while preserving the scalar fallback
for malformed NaN logits. The kernel smoke now explicitly covers ties across
SIMD lanes and vector tails.

The current 16-worker English-page smoke was **29.751 ms** tiny,
**60.676 ms** small, and **226.997 ms** medium
(`build/bench_argmax_single_pass_20260813.json`); timing remains host-noisy,
so this record is evidence of correctness rather than a broad end-to-end
speed claim. All three tiers retained 3/3 ORT decoded-text parity on English,
ultra-wide, and low-contrast samples
(`build/accuracy_argmax_single_pass_core_20260813.json`), and the Radeon
780M Vulkan smoke passed.

### 2026-08-13: diverse-shape correctness and terminal-CTC admission recheck

The current CPU implementation was additionally checked against ONNX Runtime
on `en.ppm`, the `3000x80` ultra-wide strip, and `noisy_lowcontrast.ppm` for
all three model tiers.  Decoded line count and text parity passed **3/3** for
tiny, small, and medium; the durable report is
`build/accuracy_diverse_core_20260813.json`.  A short stress smoke on the
low-contrast page and a `80x3000` ultra-tall strip also exercised the
opposite detector resize branch, where it found 3 and 20 text regions,
respectively, for every tier.  The narrow vertical strip does not yet have
text parity with this repository's ORT reference, so it is retained as a
throughput/robustness sample rather than being claimed as a correctness gate.

The bounded terminal-GEMM/CTC path was then A/B measured with three warm-ups
and 15 timed iterations per normal and ultra-wide input.  It improves tiny on
this host (English page **35.105 -> 31.379 ms**) but is neutral/negative for
small and slower for medium's ultra-wide case (**145.585 -> 276.362 ms**).
It therefore remains explicit opt-in via
`PPOCR_ENABLE_FUSED_TERMINAL_CTC=1`; the default executor remains the faster
cross-tier choice.  Raw A/B data: `build/bench_terminal_ctc_ab_20260813.json`.

### 2026-08-13: medium detector AVX-512 tile-pressure recheck

The medium-detector profile identifies regular `3x3` and `5x5`/`7x7`
context convolutions as the main remaining CPU hotspots.  Their eight-output
AVX-512 tiles were rechecked against the established four-output kernels. On
this client AVX-512 host, the larger tiles increase register pressure and can
produce substantial low-contrast tail latency: over eight timed iterations,
the medium `noisy_lowcontrast.ppm` median was **554.202 ms** with the default
four-output tiles versus **571.360 ms** with both eight-output experiments;
the latter's p95 reached **2569.152 ms**. The wide-strip median was likewise
**159.735 ms** (four) versus **163.488 ms** (eight).

Consequently, the proven four-output 3x3/wide-context tiles are now the
default. Eight-output variants remain opt-in for a separately measured server
CPU via `PPOCR_ENABLE_AVX512_CONV3X3_TILE8=1` and
`PPOCR_ENABLE_AVX512_CONV_WIDE8=1`; setting either `PPOCR_DISABLE_*` switch
still disables its matching experiment. Text parity remains 2/2 for tiny,
small, and medium on English and ultra-wide input; reports are
`build/accuracy_{tiny,small,medium}_avx512_tile4default_20260813.json`.

The refreshed medium CPU resolution ladder also passed. Its 4K private sample
was **149,164,032 B**, the immediate following 1080p sample was
**149,123,072 B**, and the final 64x64 sample was **148,738,048 B**. Private
bytes and working set both declined after the 4K image. Full evidence is
`build/memory_ladder_medium_cpu_avx512_tile4default_20260813.json`.

### 2026-08-13: AVX-512 2x2 SAME_UPPER detector-stem kernel

The detector's early `2x2` SAME_UPPER Conv nodes now use a dedicated AVX-512
interior kernel with explicit final-row/final-column zero-padding handling.
This avoids generic variable-kernel address arithmetic on the large feature
maps while retaining runtime AVX2/scalar fallbacks. The low-level smoke suite
checks odd/even dimensions, vector/scalar tails, and fused ReLU. Decoded text
parity against ONNX Runtime passed **2/2** (`en.ppm`, `wide_strip_3000x80`) for
tiny, small, and medium; reports are
`build/accuracy_{tiny,small,medium}_avx512_same2x2_20260813.json`.

On the local AVX-512 host the A/B is strongly workload dependent because the
shared machine has substantial scheduling variance. Repeated medium samples
put the specialized median around **181.580閳?32.008 ms**, versus
**187.559閳?08.236 ms** with the generic path; tiny/small consistently improved
on the short sample. It remains enabled because it removes the generic hot
loop, passes the complete correctness gates, and keeps a one-variable runtime
escape hatch for deployments whose AVX-512 frequency/cache balance differs.

### 2026-08-13: current ascending-resolution memory recovery measurement

`bench/generate_memory_ladder.py` generates deterministic PPM inputs from
`16x16` through `3840x2160`, including wide and tall 1,048,576-pixel strips.
The sequence then processes `1920x1080`, both strips again, and a final
`64x64` image. `ppocr_mem_soak` keeps one OCR instance alive, releases each
decoded image before sampling, and records Windows private bytes and working
set after every inference.

The current medium CPU two-cycle measurement passed all bounded-return gates.
In cycle 1, the `3840x2160` sample was **146,067,456 B** private /
**150,147,072 B** working set; the final `64x64` sample returned to
**143,634,432 B** / **147,345,408 B**, below the 4K sample in both measures.
Cycle 2 similarly returned from **146,886,656 B** / **150,986,752 B** at 4K
to **146,767,872 B** / **150,581,248 B** at the final small image. The first
post-4K 1080p sample declined in both measures in cycle 2; cycle 1 retained a
small amount of reusable allocator capacity before declining by the final
small-image checkpoint. This is a single-host allocator measurement, not an
assertion that an OS must decommit capacity immediately.

Use `--require-post-large-decline` to make immediate private-byte *and*
working-set decline a strict CI gate when that stronger condition is required;
the default remains bounded return because Windows allocator behavior is
non-deterministic. The complete per-resolution record is
`build/memory_ladder_medium_cpu_current_bounded_recovery_20260813.json`.

### 2026-08-13: verified current resolution-memory regression

The reproducible small-to-large-to-small test remains part of the native
regression suite.  It generates and processes, in order,
`16x16 -> 64x64 -> 160x120 -> 320x240 -> 640x480 -> 1280x720 -> 4096x256 ->
256x4096 -> 1920x1080 -> 2560x1440 -> 3000x1600 -> 3840x2160 -> 1920x1080
-> 4096x256 -> 256x4096 -> 64x64`.  Every PPM is decoded, inferred, and
destroyed before collecting Windows private bytes and working set, while one
OCR instance remains alive.  This tests both varied resolution growth and
whether capacity is bounded after a large page.

On the current medium CPU build (16 kernel workers), the fresh two-cycle run
passed all 20% bounded-return gates.  Cycle 1 returned from the 4K sample's
**147,230,720 B** private / **150,204,416 B** working set to
**143,486,976 B** / **147,460,096 B** at final `64x64`.  Cycle 2 showed an
immediate 4K-to-1080p decline in both measures (**148,221,952 B** /
**152,256,512 B** to **147,570,688 B** / **151,240,704 B**); its final small
sample remained below its pre-return peak.  The immediate-decline condition is
reported separately because Windows heap reclamation is allocator-run
dependent; the default regression correctly gates bounded recovery rather than
requiring decommit on every run.  Full per-resolution evidence:
`build/memory_ladder_medium_cpu_current_resolution_recovery_20260813.json`.

### 2026-08-14: exact-size preprocessing fast path

The fused RGB-to-BGR/NCHW preprocessor now recognizes an exact-size resize
region.  In that case bilinear interpolation is an identity, so it writes the
three normalized planes directly, without a horizontal-coordinate table or
four-pixel interpolation work per output value.  It preserves recognizer
right-padding because it writes only the natural-width columns.  The general
bilinear path remains unchanged for every resized image.  For deployment A/B,
set `PPOCR_DISABLE_IDENTITY_RESIZE_FASTPATH=1` before creating an OCR object.

The direct and generic paths produced identical native demo output for tiny,
small, and medium on exact-size `line_en`, `line_zh`, and `single_word`
inputs (`build/identity_resize_cpp_equivalence_20260814.json`).  Each model
also retained ORT decoded-text parity on an exact-size line, normal English
page, and 3000x80 ultra-wide page (3/3 each;
`build/accuracy_identity_resize_cached_{tiny,small,medium}_20260814.json`).
Kernel and Radeon 780M Vulkan smoke passed.

A fresh medium CPU resolution ladder remained bounded after the change: the
4K sample was **148,512,768 B** private / **152,322,048 B** working set and
the final `64x64` sample was **148,197,376 B** / **151,797,760 B**, below the
pre-return peak in both metrics.  Full record:
`build/memory_ladder_medium_cpu_identity_resize_20260814.json`.

The local 16-worker, CPU-only 100-run line benchmark was host-noisy on tiny,
but gave a stable medium improvement: **72.046 ms -> 67.421 ms** mean
(**6.42%** reduction) and **71.770 ms -> 67.466 ms** median.  Small and tiny
are already dominated by model execution on this single-line workload, so
this is recorded as a targeted preprocessing win rather than a claimed
uniform end-to-end speedup.  Raw A/B record:
`build/bench_identity_resize_cached_ab_20260814.json`.

### 2026-08-14: recognizer average-pool compaction

All three PP-OCRv6 recognizers contain the same valid `AveragePool(3x2,
stride=3x2)` bridge between convolutional features and the transformer.  A
dedicated NCHW kernel now executes that exact non-overlapping geometry, and
when its input has a graph-proven final use it compacts the result into the
front of the existing activation allocation before resizing it.  This removes
one bridge-output allocation and releases the unused capacity before the
transformer.  `PPOCR_DISABLE_AVERAGE_POOL3X2_FASTPATH=1` and
`PPOCR_DISABLE_AVERAGE_POOL3X2_INPLACE=1` are narrow A/B fallbacks.

The in-place route produced identical native demo output for tiny, small, and
medium on `line_en`, `line_zh`, `single_word`, and `en`
(`build/average_pool3x2_inplace_cpp_equivalence_20260814.json`). Each model
also retained ORT decoded-text parity on an exact-size line, English page, and
ultra-wide strip (3/3 each;
`build/accuracy_averagepool_inplace_{tiny,small,medium}_20260814.json`).
The current medium resolution-memory ladder passed: it returned from the
pre-return private/working-set peak of **149,241,856 B / 153,153,536 B** to
**143,618,048 B / 147,505,152 B** at final `64x64`.
`build/memory_ladder_medium_cpu_averagepool_inplace_20260814.json` preserves
every resolution sample.

On the local 16-worker CPU, 100-run one-line A/B was positive for medium
(**69.535 -> 68.032 ms**, 2.16%) and within shared-host noise for tiny/small.
It is therefore a verified allocation reduction and a targeted medium
throughput gain, not a uniform speed claim. Raw results:
`build/bench_averagepool_inplace_ab_20260814.json`. The Radeon 780M Vulkan
smoke and hybrid batch parity passed; hybrid continues to admit a Vulkan
segment only after its complete H2D, dispatch, fence, D2H, and CPU comparison
is no slower.

### 2026-08-14: attention QKV vector-copy transpose

The small and medium recognizer attention blocks export a QKV projection as
`[N,T,3,H,D]`, followed by the fixed ONNX permutation
`[2,0,3,1,4] -> [3,N,H,T,D]`.  The innermost head feature vector `D` is
contiguous before and after that permutation.  The executor now copies this
vector with `memcpy` rather than recalculating five-dimensional scalar
coordinates for every float.  All other transposes retain the generic path.
Set `PPOCR_DISABLE_QKV_TRANSPOSE_FASTPATH=1` before constructing OCR to A/B
the generic route.

Decoded-text parity against ONNX Runtime passed on English, Chinese, and
3000x80 ultra-wide inputs for tiny, small, and medium (3/3 for each tier;
`build/accuracy_qkv_transpose_{tiny,small,medium}_20260814.json`). Tiny has
no QKV attention block and remains unaffected. On the local AVX-512 CPU,
the 40-run English-page sample measured small **74.557 -> 70.835 ms**
(5.0%) and medium **217.134 -> 199.390 ms** (8.2%). On the distinct
30-run ultra-wide workload it measured small **44.199 -> 41.999 ms** (5.0%)
and medium **141.816 -> 139.679 ms** (1.5%). The one page measurements are
host-specific; raw records are `build/bench_qkv_transpose_ab_20260814.json`
and `build/bench_qkv_transpose_wide_ab_20260814.json`.

The current medium CPU small-to-4K-to-small ladder stayed bounded: the final
64x64 sample was **143,474,688 B** private / **147,525,632 B** working set,
below both the 4K sample (**146,661,376 B / 150,781,952 B**) and the
pre-return peak.  Full per-resolution evidence is
`build/memory_ladder_medium_cpu_qkv_transpose_20260814.json`.

### 2026-08-14: QKV transpose/slice elimination

The small and medium attention exports immediately split the just-transposed
`[3,N,H,T,D]` QKV activation into three `Slice(1) -> Squeeze(0)` branches.
The loader now recognizes this complete three-branch topology and replaces it
with three direct copies from the original `[N,T,3,H,D]` projection into their
already-squeezed `[N,H,T,D]` destinations. This eliminates the full
transposed QKV activation and the three temporary slice buffers. It is a
strict model-shape fusion; all general Transpose/Slice graphs retain their
ordinary execution. `PPOCR_DISABLE_QKV_SLICE_FUSION=1` is the deployment A/B
fallback.

On the local AVX-512 CPU, the 20-run English-page A/B, with the earlier
vector-copy transpose still enabled in the baseline, measured small
**76.315 -> 72.245 ms** (5.3%) and medium **222.274 -> 195.827 ms** (11.9%).
The baseline values include the prior QKV-vector optimization, so this records
the additional benefit of removing the full transpose/slice lifetime rather
than double-counting it. Raw record:
`build/bench_qkv_slice_fusion_ab_20260814.json`.

The default fused route retained ORT decoded-text parity on English, Chinese,
and 3000x80 ultra-wide input for all three shipped model tiers (3/3 each;
`build/accuracy_qkv_slice_fusion_default_{tiny,small,medium}_20260814.json`).
Tiny has no matching transformer QKV topology and is unchanged. The medium
ascending-resolution memory gate passed: final `64x64` was
**143,544,320 B** private / **151,076,864 B** working set, below the 4K
sample and pre-return peak. Full record:
`build/memory_ladder_medium_cpu_qkv_slice_fusion_20260814.json`. CPU kernel
and Radeon 780M Vulkan smoke also passed; hybrid admission policy remains
unchanged and selects only complete GPU segments that are no slower than CPU.

### 2026-08-14: complete QKV split lifetime and resolution-memory recheck

The three direct Q/K/V copies are now emitted by one `FusedQkvSplit` node
with three outputs.  This keeps the shared `[N,T,3,H,D]` projection live only
until all three final `[N,H,T,D]` tensors have been materialized, then retires
it through the normal use counter.  It is a PP-OCRv6 topology-specific memory
fusion, not general multi-output ONNX runtime support.

The existing native resolution ladder provides the requested ascending image
coverage: `16x16`, `64x64`, `160x120`, `320x240`, `640x480`, `1280x720`,
wide/tall `4096x256` and `256x4096`, `1920x1080`, `2560x1440`, `3000x1600`,
and `3840x2160`, followed by a staged recovery tail and `64x64` return. A
fresh two-cycle medium CPU run passed its 20% bounded-recovery gate. In cycle
1, private / working-set memory declined from **149,700,608 / 153,661,440 B**
at 4K to **147,795,968 / 151,752,704 B** at the immediately following 1080p
page, then finished at **147,509,248 / 151,162,880 B** on the return 64x64.
Cycle 2 likewise declined immediately after 4K; the final small sample was
**147,435,520 / 151,105,536 B**, only **0.27%** above the warmed private-byte
baseline.  Per-resolution evidence is
`build/memory_ladder_medium_cpu_qkv_split_20260814.json`. Windows allocator
capacity is reusable and samples may vary, so the gate verifies bounded
recovery rather than demanding that every allocation be decommitted.

The fused executor also retained decoded-text parity with ONNX Runtime on
English, Chinese, and 3000x80 ultra-wide inputs (3/3 medium cases;
`build/accuracy_qkv_split_default_medium_20260814.json`). CPU kernel and
Radeon 780M Vulkan smoke tests passed after the change.

A final all-tier recheck repeated those three text/diverse-size cases for
tiny, small, and medium (**3/3 each**):
`build/accuracy_qkv_split_{tiny,small,medium}_20260814.json`. The 20-run
CPU snapshot (16 workers, exact GELU) was tiny **30.920 / 21.393 ms**, small
**62.485 / 44.688 ms**, and medium **198.325 / 132.951 ms** for English /
3000x80, respectively (`build/bench_cpu_diverse_qkv_split_20260814.txt`).
The former is timing evidence for this host only; the durable cross-engine
comparison remains the longer, independently measured CPU-versus-ORT corpus.
The final one-cycle 16-sample medium ladder also passed its bounded-return
gate (`build/memory_ladder_medium_cpu_qkv_split_final_20260814.json`), with
the return `64x64` footprint below the pre-return peak. Its immediate 4K
sample did not decline in that isolated Windows run, so the report records
that fact rather than claiming forced allocator decommit.

### 2026-08-14: extended 8K source-image recovery gate

The generated memory corpus now continues beyond 4K with `5120x2880` and
`7680x4320` pages, then immediately processes a `3840x2160` recovery page
before the existing 1080p, extreme-aspect-ratio, and final-`64x64` tail. The
native soak keeps one OCR instance alive but scopes every decoded RGB image to
`load -> Recognize -> destroy -> sample`; this specifically tests source image
release as well as activation reuse. It is intentionally more demanding than
the detector's 960-pixel inference cap, because the decoded 8K source itself
is about 95 MiB.

With medium CPU PP-OCRv6, one strict cycle passed with
`--require-post-large-decline`: private / working-set memory fell from
**149,139,456 / 153,059,328 B** after 8K to **146,808,832 / 150,470,656 B**
at the immediate 4K recovery (**-2,330,624 B / -2,588,672 B**). The final
`64x64` was **147,595,264 / 151,126,016 B**, below the pre-return peaks
(**149,151,744 / 153,059,328 B**) and only **3.03%** above the warmed private
baseline. Full 19-resolution evidence is
`build/memory_ladder_medium_cpu_8k_strict_20260814.json`. This is a
host-specific bounded-recovery observation; it does not promise every Windows
heap or Vulkan driver immediately decommits reusable pages.

### 2026-08-14: reproducible small-to-8K memory ladder

`bench/generate_memory_ladder.py` now creates 19 deterministic P6 PPM pages
in ascending source resolution from `16x16` through `5120x2880` and
`7680x4320`, followed by a `3840x2160 -> 1920x1080 -> wide/tall -> 64x64`
recovery tail. `bench/memory_ladder_regression.py` keeps one native OCR
instance alive, destroys each decoded source image immediately after
recognition, and records private bytes plus working set after every page.

The strict gate compares the 5K/8K high-water with the deliberate post-8K
recovery tail (`4K -> 1080p -> wide/tall`), and independently bounds the
final `64x64` footprint. Windows allocator/working-set accounting is not
guaranteed to be monotonic across adjacent samples, so an optional
`--require-immediate-post-large-decline` diagnostic is available, but the
default strict test requires both counters to fall below the large-page
high-water somewhere in that recovery tail. This avoids presenting sampling
jitter as a leak while still proving that the process returns from a large
page under the same live OCR instance.

### 2026-08-14: recognition-worker lifetime hardening

The production CTC executor no longer stores its compact use-counter vector
or node-input pointer scratch vector in `thread_local` storage.  Those two
small vectors are now invocation-local (the pointer list reserves eight
entries).  This is deliberately a lifetime fix rather than an allocator
cache: on the MinGW/Windows build, CTC execution can run on short-lived outer
recognition workers, and emulated-TLS vector destructors may otherwise be
registered through a different pthread DLL during worker teardown.  The
result was an intermittent `0xC0000374` heap-corruption process exit after a
successful long memory ladder.  Activation tensors themselves remain
per-invocation and are released by the existing graph use-count planner.

The current medium CPU 19-resolution ladder completed and exited cleanly.
Private / working-set memory fell from **149,811,200 / 153,808,896 B** on the
8K source page to **147,935,232 / 151,924,736 B** on its immediate 4K
recovery page; the final `64x64` sample was **143,974,400 / 147,787,776 B**,
only **0.45%** above the warmed private-byte baseline and below the complete
large-page high-water marks.  Full record:
`build/memory_ladder_medium_cpu_8k_tls_fix_20260814.json`.

CPU and hybrid runs both completed the varied `16x16`, English, 3000x80
ultra-wide, and 80x3000 ultra-tall corpus for tiny/small/medium (**12/12 per
backend**): `build/accuracy_tls_lifetime_fix_{cpu,hybrid}_diverse_20260814.json`.
Kernel smoke and Radeon 780M Vulkan smoke also passed. Hybrid retains its
strict per-shape whole-boundary no-slower admission rule; this change does not
claim that the incomplete device-resident OCR graph supports GPU-only mode.

### 2026-08-14: fresh small-to-large memory regression

The checked-in generator and regression gate were rerun against the medium
PP-OCRv6 CPU pipeline.  The deterministic corpus contains 19 PPM inputs:
`16x16 -> 64x64 -> 160x120 -> ... -> 5120x2880 -> 7680x4320`, followed by
`3840x2160 -> 1920x1080 -> wide/tall strips -> 64x64`.  It therefore measures
both normal increasing resolutions and source-image release after the 8K page,
while retaining one OCR instance for the whole sequence.

The strict gate passed for both Private Bytes and Working Set.  The 8K sample
was **148,996,096 / 152,911,872 B**; the immediately following 4K recovery
sample was **147,918,848 / 151,826,432 B** (**-1,077,248 / -1,085,440 B**).
The final 64x64 sample was **143,548,416 / 147,689,472 B**, only **0.25%**
above the warmed Private-Bytes baseline and below the pre-return high-water
mark.  The complete per-resolution record is
`build/memory_ladder_medium_cpu_strict_final_20260814.json`.

Reproduce it with:

```powershell
python bench/generate_memory_ladder.py --output build/memory_ladder_images
python bench/memory_ladder_regression.py --cpp build/ppocr_mem_soak.exe --det D:/workprj/aicoder/.tmp/ocr-models/ppocrv6_medium_det.onnx --rec D:/workprj/aicoder/.tmp/ocr-models/ppocrv6_medium_rec.onnx --dict build/dict_ppocrv6_latin.txt --images build/memory_ladder_images --cycles 1 --backend cpu --require-post-large-decline --report build/memory_ladder_medium_cpu_strict.json
```

### 2026-08-14: current resolution-memory validation

The reproducible ladder was generated again and run with one live medium CPU
OCR instance. It records Private Bytes and Working Set after every one of the
19 pages, with decoded PPM storage destroyed before each sample. The run
passed the bounded-return gate:

| Checkpoint | Private Bytes | Working Set |
| --- | ---: | ---: |
| warmed baseline | 143,437,824 B | 閳?|
| 8K (`7680x4320`) | 148,480,000 B | 152,453,120 B |
| first 4K recovery | 148,000,768 B | 151,769,088 B |
| final `64x64` | 146,055,168 B | 150,097,920 B |

Both counters fell immediately after 8K in this run; the final small image was
**1.82%** above the warmed private-byte baseline and below the complete
pre-return peak. The full per-resolution evidence, including the recovery
tail decision, is
`build/memory_ladder_medium_cpu_strict_current_20260814.json`. Results are
machine/run-specific process accounting, not a fixed cross-platform memory
guarantee.

### 2026-08-14: revalidated varied-resolution memory recovery

The 19-image ascending-and-recovery ladder was regenerated and rerun with one
medium PP-OCRv6 CPU instance (`PPOCR_BENCH_THREADS=16`).  Each PPM is loaded,
recognized, destroyed, and only then sampled, so source RGB storage from a
large page cannot be hidden by retaining the input vector.  The strict
post-large recovery gate passed for both Windows Private Bytes and Working Set:

| Checkpoint | Private Bytes | Working Set |
| --- | ---: | ---: |
| warmed baseline | 142,888,960 B | n/a |
| 5K peak (`5120x2880`) | 148,463,616 B | 152,395,776 B |
| 8K (`7680x4320`) | 146,882,560 B | 151,015,424 B |
| first 4K recovery | 147,435,520 B | 151,506,944 B |
| final `64x64` | 146,251,776 B | 150,216,704 B |

The observed peak was the 5K page rather than 8K, which is valid for Windows
process accounting and the detector's bounded tensor resize.  Both counters
were already below that high-water mark at the first 4K recovery sample; the
final small image remained below it and was 2.35% above the warmed private-byte
baseline.  The machine-specific, per-resolution JSON evidence is
`build/memory_ladder_medium_cpu_strict_revalidated_20260814.json`.

### 2026-08-14: zero-copy top-level ONNX protobuf traversal

Model construction no longer copies the complete embedded ONNX `GraphProto`
before parsing it.  The model reader now exposes bounded protobuf byte spans;
the top-level graph is parsed directly from the already-owned model file buffer.
Nested node, attribute, initializer, and operator-set messages retain their
existing owning parse paths, so the supported PP-OCRv6 graph semantics and
runtime's no-ONNX-Runtime dependency remain unchanged.  This removes one
transient full-model allocation/copy during startup and lowers model-loading
work without altering per-inference numerical operations.

On the local Windows AVX-512 host (`PPOCR_BACKEND=cpu`, 16 kernel workers,
one warm-up and five English-page measurements), constructor-to-ready load
time changed as follows:

| Model | Before | After | Lower load time |
| --- | ---: | ---: | ---: |
| tiny | 27.184 ms | 17.808 ms | 34.5% |
| small | 102.254 ms | 72.066 ms | 29.5% |
| medium | 519.523 ms | 294.587 ms | 43.3% |

The per-model evidence is
`build/bench_model_load_span_parse_ab_20260814.json`; the post-change
inference snapshot is `build/bench_model_load_span_parse_20260814.json`.
Kernel and Vulkan admission smoke passed. CPU and hybrid also completed the
five-size `16x16`, English, low-contrast, ultra-wide, and ultra-tall corpus
for tiny/small/medium (30 successful executions per backend); the recorded
outputs are `build/smoke_span_parse_diverse_{cpu,hybrid}_20260814.json`.
One medium ultra-wide confidence differs only in the last reported digits in
hybrid because its transfer-inclusive channel-affine/depthwise-Swish admission
can select a numerically valid Vulkan segment; decoded text and geometry are
unchanged. GPU-only remains deliberately rejected until the complete graph is
device-resident.

### 2026-08-14: invocation-local executor input scratch reuse

Both the general graph executor and the production CTC executor now reuse one
invocation-local, non-owning node-input pointer vector. Previously, the general
path allocated a separate small vector at each generic operator and at several
in-place fast paths. The new scratch has eight reserved entries (larger future
operator signatures still grow it safely), is never shared across OCR calls,
and retains the existing no-TLS lifetime rule for MinGW worker teardown.

This removes repeated allocator traffic without changing tensors, reduction
order, CPU SIMD dispatch, or the Vulkan admission policy. A fresh CPU English
snapshot (`PPOCR_BENCH_THREADS=16`, two warm-ups, twelve runs) measured
**30.817 / 61.215 / 182.730 ms** for tiny/small/medium. The varied-shape CPU
snapshot additionally completed ultra-wide at **19.090 / 44.437 / 127.590 ms**
and dense `1920x1080` at **265.831 / 1133.132 / 4880.140 ms** (three-run
shared-host samples, therefore directional rather than portability claims).

The 19-resolution medium memory ladder passed again: its 5K high-water was
**149,815,296 / 153,812,992 B** (Private Bytes / Working Set), the first 4K
recovery was **146,288,640 / 150,216,704 B**, and final `64x64` was
**146,501,632 / 150,388,736 B**, still below the pre-return high-water. The
complete evidence is `build/memory_ladder_medium_cpu_input_scratch_reuse_20260814.json`.
All 15 tiny/small/medium varied-size cases completed in both CPU and hybrid;
their decoded text and geometry agree across backends. Kernel and Radeon 780M
Vulkan admission smokes passed, with channel-affine Swish and depthwise-Swish
still selected only when measured full-boundary GPU cost is no slower than CPU.
Records: `build/bench_input_scratch_reuse_{en,diverse}_20260814.json` and
`build/smoke_input_scratch_reuse_diverse_{cpu,hybrid}_20260814.json`.

### 2026-08-14: small-to-large image memory recovery regression

The repository includes a native process-memory regression for long-lived OCR
services. `bench/generate_memory_ladder.py` produces deterministic PPM pages
that increase from `16x16` through `7680x4320`, including wide and tall
aspect-ratio cases, then process a staged `4K -> 1080p -> wide/tall -> 64x64`
recovery tail. `bench/memory_ladder_regression.py` keeps one OCR instance alive
and records Windows Private Bytes and Working Set after every resolution.

Run the full test with:

```powershell
$env:PPOCR_BENCH_THREADS = '16'
python bench/generate_memory_ladder.py --output build/memory_ladder_images
python bench/memory_ladder_regression.py `
  --cpp build/ppocr_mem_soak.exe `
  --det D:/workprj/aicoder/.tmp/ocr-models/ppocrv6_medium_det.onnx `
  --rec D:/workprj/aicoder/.tmp/ocr-models/ppocrv6_medium_rec.onnx `
  --dict build/dict_ppocrv6_latin.txt `
  --images build/memory_ladder_images --cycles 1 --backend cpu `
  --require-post-large-decline `
  --report build/memory_ladder_medium_cpu_compact_uses_20260814.json
```

On the current Windows x86 host, the test passed after the compact executor
use-count change. The `7680x4320` sample reached **150,474,752 B / 154,353,664
B** (Private Bytes / Working Set). The recovery tail fell to **147,111,936 B /
151,142,400 B** at `1920x1080`, and the final `64x64` checkpoint was
**148,066,304 B / 151,736,320 B**. Thus both counters declined below the
post-large high-water without restarting the OCR instance. The adjacent first
4K recovery measurement may briefly exceed the 8K sample because of Windows
allocator/working-set accounting; the enforced gate intentionally verifies a
decline in the complete post-large recovery tail instead of claiming a brittle
single-sample monotonic property. Full per-resolution evidence is
`build/memory_ladder_medium_cpu_compact_uses_20260814.json`.

### 2026-08-14: refreshed exact detector batching across three model sizes

The true same-shape detector NCHW batch was rerun after the memory work with
four copies of the English page. The check compares every text, quadrilateral,
bounding box, and confidence against four serial calls. Recognition retained
`rec_batch_size=1`: an investigation confirmed that forcing four recognizer
crops through the transformer can change exact CTC confidence on this host, so
it is not used for the exact-output benchmark.

With `PPOCR_BENCH_THREADS=16`, `PPOCR_DET_BATCH_SIZE=4`,
`PPOCR_REC_BATCH_SIZE=1`, and `PPOCR_BATCH_PREPROCESS_PARALLELISM=4`, the
three-run means were:

| Model | CPU serial | CPU batch | CPU speedup | Hybrid serial | Hybrid batch | Hybrid speedup |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| tiny | 133.272 ms | 76.211 ms | 1.749x | 157.468 ms | 128.652 ms | 1.224x |
| small | 282.395 ms | 177.486 ms | 1.591x | 442.434 ms | 390.907 ms | 1.132x |
| medium | 1241.956 ms | 1013.146 ms | 1.226x | 1230.259 ms | 1138.408 ms | 1.081x |

The corresponding exact-parity record is
`build/batch_detector_rec1_current_20260814.json`. Hybrid remains an
honest per-segment policy: Vulkan is selected only when the full measured
host-to-device, dispatch, and device-to-host boundary is no slower than CPU;
on this Radeon 780M run, small channel-affine Swish and depthwise-Swish
segments met that rule while the remainder correctly stayed on SIMD CPU.

### 2026-08-14: verified four-page detector NCHW batching

`RecognizeBatch()` was rechecked on four same-shape English pages for every
PP-OCRv6 tier.  The detector receives one real `[4,3,H,W]` tensor; each
result, including text, quadrilateral, bounding box, and confidence, was
checked against four serial `Recognize()` calls by `ppocr_batch_smoke`.
Recognition remains `rec_batch_size=1` in this comparison: packing unequal
recognizer widths by right-padding changes transformer attention and can alter
CTC confidence, so it is not represented as exact-output acceleration.

With 16 CPU workers, `det_batch_size=4`, four preprocessing workers, one
warm-up, and five measured CPU iterations, serial-to-batch time was:

| Model | Serial mean | Batched mean | Throughput gain |
|---|---:|---:|---:|
| tiny | 146.914 ms | 86.413 ms | 1.700x |
| small | 318.376 ms | 204.255 ms | 1.559x |
| medium | 1289.731 ms | 1011.596 ms | 1.275x |

The corresponding hybrid recheck (three iterations; the same exact-output
gate) measured **1.218x / 1.138x / 1.079x** for tiny/small/medium. Hybrid
still admits each Vulkan segment only after its complete H2D + shader + D2H
path is no slower than the matching CPU path; this is not a claim that the
incomplete full graph supports `gpu_only`. The host-specific full record is
`build/bench_batch_detector_nchw_current_20260814.json`.

Reproduce the CPU measurement with:

```powershell
$env:PPOCR_BACKEND='cpu'; $env:PPOCR_BENCH_THREADS='16'
$env:PPOCR_DET_BATCH_SIZE='4'; $env:PPOCR_REC_BATCH_SIZE='1'
$env:PPOCR_BATCH_PREPROCESS_PARALLELISM='4'; $env:PPOCR_BATCH_WARMUP='1'; $env:PPOCR_BATCH_RUNS='5'
.\build\ppocr_batch_smoke.exe DET.onnx REC.onnx dict.txt 1 en.ppm en.ppm en.ppm en.ppm
```

### 2026-08-13: Vulkan ConvTranspose + Add primitive groundwork

The Vulkan backend now has a numerically tested `ConvTranspose(2x2,
stride=2) + equal-shape Add` primitive. It keeps the projection in device
buffer D, inserts a compute write/read barrier, adds the residual in a second
dispatch, and copies only the final sum back. Its hybrid admission is strict:
it measures and validates complete `H2D + two dispatches + fence + D2H`
against the matching CPU SIMD sequence, then selects Vulkan only when no
slower.

On this Radeon 780M test shape (`N=4, Cin=32, Cout=40, 31x29`), GPU was
**2.771 ms** versus CPU **0.535 ms**, so CPU correctly remains selected. This
is backend coverage, not an end-to-end GPU speed claim. The current PP-OCRv6
FPN export folds its following `[1,C,1,1]` Add into the transpose bias, so the
new equal-shape-residual primitive is intentionally not claimed as active for
that model revision. `ppocr_vulkan_smoke` covers it; tiny/small/medium CPU
text parity passed **2/2** each on `en.ppm` and `wide_strip_3000x80.ppm`,
recorded in `build/accuracy_{tiny,small,medium}_convtranspose_add_20260813.json`.

## Build

### 2026-08-14: compact execution liveness and resolution-memory revalidation

The detector's generic executor now resets compact `uint16_t` use counters
indexed once at model load, rather than cloning a string-keyed use-count map
for every page. This removes per-inference hash-table bookkeeping while
retaining the existing last-use tensor release rules. The change applies only
to the fixed PP-OCRv6 graph topology; arbitrary graph values still retain
their original tensor ownership and output semantics.

The CPU multi-size smoke completed tiny/small/medium over five distinct
shapes (`16x16`, English, ultra-wide, ultra-tall, and dense `1920x1080`). The
record is `build/bench_diverse_compact_uses_20260814.json`; its three-run
means are intentionally retained as per-host measurement evidence rather than
portable speed claims. Kernel and Radeon 780M Vulkan admission smokes also
passed. Hybrid retains its full transfer-inclusive rule: a Vulkan segment is
used only after measured `H2D + GPU + D2H` is no slower than the same CPU
segment. GPU-only remains deliberately rejected because the complete graph is
not yet device-resident.

The 19-resolution medium CPU ladder was revalidated with one OCR instance
alive throughout. It passed `--require-post-large-decline`: the `7680x4320`
sample measured **147,972,096 B / 151,748,608 B** (Private Bytes / Working
Set), then the recovery tail reached **146,042,880 B / 148,320,256 B** at the
wide `4096x256` sample. The final `64x64` checkpoint was **147,783,680 B /
151,691,264 B**, below the largest-image private-byte value and below the
pre-return high-water. Full per-resolution evidence is
`build/memory_ladder_medium_cpu_compact_uses_final_20260814.json`.

### 2026-08-14: current ascending-resolution memory regression

The same long-lived medium CPU instance was rerun over all 19 generated
images, in ascending source sizes from `16x16` to `7680x4320`, followed by
`3840x2160 -> 1920x1080 -> 4096x256 -> 256x4096 -> 64x64`.  Each page is
scoped to `load -> Recognize -> destroy -> sample`, so the measurements expose
both inference activation reuse and whether a decoded large source image stays
resident after its OCR call.

The strict recovery gate passed.  At 8K, Private Bytes / Working Set were
**148,131,840 B / 152,174,592 B**.  The immediately following 4K page was
**147,681,280 B / 151,670,784 B**, a drop of **450,560 B / 503,808 B**.  The
final `64x64` checkpoint was **147,587,072 B / 151,429,120 B**, below both the
8K sample and the pre-return high-water (**150,106,112 B / 154,062,848 B**),
without recreating the OCR instance.  Per-resolution samples and the enforced
pass/fail decision are recorded in
`build/memory_ladder_medium_cpu_current_resolution_recovery_20260814.json`.


### 2026-08-14: refreshed three-tier CPU / Hybrid comparison

`bench/compare.ps1` now accepts `-Backend cpu|hybrid|gpu` and passes the
choice through to the native benchmark. This makes the CPU and Hybrid runs
explicit in the same ONNX Runtime comparison harness instead of inheriting an
ambient process environment.

On the local AVX-512 host, `en.ppm`, 2 warm-ups and 6 measured runs, the
native CPU implementation measured **32.315 ms** (tiny) and **67.419 ms**
(small), versus ONNX Runtime CPUExecutionProvider **113.453 ms** and
**149.804 ms**: **71.5%** and **55.0%** lower mean latency respectively. The
same harness measured medium Hybrid at **239.765 ms** versus ONNX Runtime
**333.395 ms** (**28.1%** lower). These are machine/run-specific samples;
reproduce before treating them as deployment guarantees.

The medium high-density `dense_blobs` stress bound was refreshed to
`[100, 3600]`. The current native medium detector produced 2,903 valid
components and the full 11-image varied-size stress suite passed.

### 2026-08-14: terminal CTC projection is now the default

The recognizer's final vocabulary projection is now executed through the
bounded `GemmCtcTop1` route by default. It computes each small time-step tile,
performs the CTC argmax/probability reduction, and releases that tile rather
than materializing the full `[N,T,V]` logit tensor. Set
`PPOCR_DISABLE_FUSED_TERMINAL_CTC=1` only for an A/B comparison with the older
materialized-logit route.

On the dense `ui_dense_1920x1080` corpus the default route reduced measured
CPU latency in this local run from **279.538 to 263.995 ms** for tiny
(123 crops), **1077.122 to 965.587 ms** for small (119 crops), and
**6285.383 to 5621.346 ms** for medium (122 crops). These samples are
machine-specific; text parity against ONNX Runtime passed 4/4 varied inputs
for each model after enabling the default.

### 2026-08-14: refreshed 16x16-to-8K memory recovery measurement

The deterministic 19-image ladder was rerun with one live medium CPU OCR
instance. It processes ascending source resolutions from `16x16` through
`7680x4320`, then a `3840x2160 -> 1920x1080 -> 4096x256 -> 256x4096 ->
64x64` recovery tail. Every input is scoped to `load -> Recognize -> destroy
-> sample`, so this measures both activation reuse and release of a large
decoded RGB source image.

The strict post-large recovery gate passed. The 5K/8K high-water was
**150,675,456 B / 154,615,808 B** (Private Bytes / Working Set); the first
4K recovery sample was **150,036,480 B / 154,087,424 B**, and the final
`64x64` sample was **148,877,312 B / 151,830,528 B**. The final small-image
footprint was 3.63% above the warmed private-byte baseline and below both
pre-return peaks, confirming bounded memory recovery without recreating the
OCR instance. Full per-resolution evidence is
`build/memory_ladder_medium_cpu_post_large_20260814.json`. Windows allocator
accounting may fluctuate between adjacent samples; the enforced condition is
that both counters return below the 5K/8K high-water somewhere in the
post-large recovery tail.
### 2026-08-14: CTC tile-policy validation and final memory recheck

The four-row bounded terminal-CTC projection remains the default. An
experimental automatic eight-row choice for the wider recognizers was tested
on the high-density `1920x1080` sample, but was slower in the paired local
runs: small measured **914.427 ms** versus **897.369 ms** with four rows, and
medium measured **4829.971 ms** versus **4529.301 ms**. The larger tile
remains available only through `PPOCR_FUSED_TERMINAL_CTC_TILE_ROWS` for
platform-specific experiments; it is not used as the portable default.

After this decision, the complete 19-resolution medium CPU memory check
passed again. Private Bytes / Working Set fell from **147,623,936 B /
151,322,624 B** at the 8K sample to **146,046,976 B / 149,807,104 B** at the
first 4K recovery; final `64x64` was **143,917,056 B / 147,906,560 B**
(**0.04%** above the warmed private-byte baseline), below the full pre-return
high-water. The durable per-resolution record is
`build/memory_ladder_medium_cpu_tile4_20260814.json`.
### 2026-08-14: recognizer hotspot profiling and Vulkan Hybrid revalidation

`PPOCR_PROFILE=1` now reports the terminal-CTC executor even when its fused
`GemmCtcTop1` path returns early. On the medium ultra-wide crop, that direct
measurement identifies the terminal vocabulary projection as the largest
recognizer operation (**40.456 ms** in this local profile), followed by the
existing fused 1x1/residual convolutions. This makes future optimization work
measurable instead of inferring recognizer cost from detector-only profiles.

The Vulkan Hybrid policy was revalidated on AMD Radeon 780M. It measures each
implemented segment as a complete synchronous `H2D + dispatch + D2H` path and
selects GPU on `gpu_mean <= cpu_mean`; smoke selected GPU for channel-affine
Swish (**0.193 vs 0.404 ms**) and depthwise-Swish (**0.341 vs 0.346 ms**),
while transfer-bound primitives correctly remained on CPU. Tiny, small, and
medium Hybrid text parity all passed on normal and ultra-wide inputs (6/6).
GPU-only remains deliberately unavailable until the complete detector,
recognizer, and post-processing graph is device-resident; it does not silently
fall back to CPU.

The AVX-512 four-row GEMM microkernel was also retested against the stable
two-row default. It remains an explicit
`PPOCR_ENABLE_AVX512_GEMM4ROW=1` deployment A/B option only: its effect varied
between otherwise similar paired runs, so it is not enabled by default.

### 2026-08-14: medium terminal-CTC projection parallelism

The terminal CTC projection now uses a bounded **32-row** vocabulary tile for
the medium recognizer (`depth >= 384`, `vocab >= 1024`); tiny and small retain
the prior four-row tile. This gives a single ultra-wide medium crop enough
independent GEMM rows to use the persistent SIMD pool, while keeping CTC
blank/repeat state serial in time order and the workspace bounded. The
`PPOCR_FUSED_TERMINAL_CTC_TILE_ROWS=1|2|4|8|16|32` override remains available
for deployment-side A/B.

On this AVX-512 Windows host (16 workers), paired ultra-wide `3000x80`
measurements improved medium from **189.408 ms** (four rows) to **150.969 ms**
(32 rows), a **20.3%** reduction; small improved from **61.882 to 56.759 ms**
in that isolated wide test, but dense-page runs were less stable, so its
conservative default remains four rows. Medium dense `1920x1080` also improved
from **5237.145 to 4671.601 ms** in the paired run. These figures are local
host measurements, not portable latency guarantees.

The associated 19-resolution medium CPU memory gate passed: 8K was
**150,437,888 B / 154,501,120 B** (Private Bytes / Working Set), immediate
4K recovery was **148,398,080 B / 152,215,552 B**, and final `64x64` was
**144,076,800 B / 148,070,400 B**, below baseline and all large-page
high-water marks. Per-resolution evidence:
`build/memory_ladder_medium_cpu_ctc32_20260814.json`.

The same gate was rerun after the final CTC policy validation. It again
passed the enforced `--require-post-large-decline` condition: the 5K/8K
high-water was **149,454,848 B / 153,169,920 B**, the first post-8K 4K page
fell to **146,534,400 B / 150,233,088 B**, and the final `64x64` checkpoint
was **147,755,008 B / 151,703,552 B**. This is **0.37%** above the warmed
**147,210,240 B** private-byte baseline and below both pre-return peaks, so
large decoded images and inference workspaces do not remain pinned after the
large-image sequence. The ladder deliberately covers 19 inputs from `16x16`
through `7680x4320`, including wide/tall aspect-ratio branches and a staged
`4K -> 1080p -> wide -> tall -> 64x64` recovery tail. Full JSON evidence:
`build/memory_ladder_medium_cpu_ctc32_recheck_20260814.json`.

### 2026-08-14: CTC workspace reuse experiment rejected

A CTC vocabulary-tile reuse experiment was measured on the 123/119/122-crop
dense `1920x1080` page. Sharing one preallocated tile per in-flight sequence
increased the medium batch workspace enough to cause a memory-recovery gate
failure on one fresh run, while its latency did not improve reliably. The
production implementation therefore retains a short-lived, per-worker bounded
tile: it has a strict `32 * vocab`-float cap for medium and `4 * vocab` for
tiny/small, releases promptly after each recognition task, and avoids unsafe
MinGW thread-local destructors. This choice is deliberately evidence-led: the
restored implementation passed the 19-resolution CPU memory gate again, with
the first recovery-tail point below the 8K private/working-set high-water at
`1920x1080` (**146,456,576 B / 149,090,304 B**) and final `64x64` still below
the pre-return peak. Evidence:
`build/memory_ladder_medium_cpu_ctc_workspace_reverted_20260814.json`.

### 2026-08-14: hardware-aware recognition worker bound

The 12-way CPU recognition default is capped at
`std::thread::hardware_concurrency()`. This retains the measured 12-way
policy on the local 16-logical-core AVX-512 machine while preventing a small
x86 or ARM target from constructing more independent crop graphs than its
available logical CPUs. Hybrid deliberately keeps its separate
`hybrid_graph_parallelism` policy because its Vulkan queue is serialized. The
public `PPOCR_REC_PARALLELISM` override remains available for host-specific
tuning.

### 2026-08-14: dense-page recognition scheduler tuning

The CPU default `rec_parallelism` is now **12** (override with
`PPOCR_REC_PARALLELISM`). A 16-worker AVX-512 sweep on the 123/119/122-crop
dense `1920x1080` page measured 8-way versus 12-way versus 16-way outer
recognition scheduling. Twelve was the stable cross-model choice: tiny
improved from **282.675 to 241.928 ms** (14.4%), small from **949.373 to
**885.617 ms** (6.7%), and medium from **3968.286 to 3905.875 ms** (1.6%).
At 16 ways medium regressed to **4375.612 ms**, so the default deliberately
leaves headroom for its larger SIMD kernels and avoids multiplying dense-page
activation lifetime further.

A fresh default run measured **236.679 / 854.346 / 4136.895 ms** for
tiny/small/medium respectively (3/3, 3/3, and 2/2 measured samples). Normal
and ultra-wide decoded-text parity against ORT remained 6/6. The 19-resolution
medium memory gate also passed: after the 8K page, Private Bytes / Working
Set fell from **149,106,688 / 153,096,192 B** to **147,845,120 /
151,638,016 B** at the first 4K recovery page; the final `64x64` checkpoint
remained below the large-page high-water. Evidence:
`build/memory_ladder_medium_cpu_recparallel12_20260814.json`.

### 2026-08-14: multi-image native/ORT gate repair and revalidation

`bench/compare_models.ps1` now reliably expands a comma-separated or native
PowerShell `string[]` image list, forwards the selected CPU/Hybrid backend,
and distinguishes a latency-weighted corpus gate from an explicit per-case
floor. This prevents a one-image invocation from being presented as a
multi-resolution result and makes each model/image pair independently auditable.

A fresh CPU validation used normal English and ultra-wide `3000x80` text,
16 workers, one warm-up, two timed runs, and required every one of the six
model/image pairs to beat ONNX Runtime CPU. All passed. The latency-weighted
native reductions were **68.1%** (tiny), **56.2%** (small), and **22.9%**
(medium); no model had a losing case in this run. Full machine-specific
evidence: `build/compare_models_cpu_normal_wide_gate_20260814.json`.

The same wrapper also supports `-Backend hybrid`; on this Radeon 780M UMA
host, it preserves the transfer-inclusive Vulkan admission policy rather than
claiming universal end-to-end GPU gains.

### 2026-08-14: refreshed three-tier ONNX Runtime comparison

The current 16-worker CPU comparison (2 warm-ups, 6 measured `en.ppm` runs)
continues to show substantial native gains over ONNX Runtime CPUExecutionProvider:
**33.084 vs 118.015 ms** for tiny (**72.0% lower**), **72.522 vs 195.894 ms**
for small (**63.0% lower**), and **222.385 vs 402.871 ms** for medium
(**44.8% lower**). These are paired local-host measurements and include the
complete native OCR pipeline, not an isolated kernel claim.

An additional selected-class CTC-softmax parallelization experiment was
rejected: it was neutral on a single wide crop and slower on a dense medium
page, so the shipped path retains the lower-variance temporal CTC scan.
Hybrid text parity was rerun for tiny/small/medium on normal and ultra-wide
inputs (6/6); the GPU decision rule remains unchanged: choose Vulkan only
when the complete transfer-inclusive segment is no slower than CPU.

### 2026-08-14: Vulkan device-resident graph foundation

The Vulkan backend now has a `VulkanTensorArena` separate from Hybrid's five
short-lived scratch bindings. It allocates FP32 storage slots by tensor size,
reuses a released slot only after the caller's final-use decision, and reports
live versus retained capacity. Graph boundaries can upload inputs/constants
and download requested outputs, while intermediate elementwise values remain
in Vulkan buffers. The first device-resident operators are exact binary
in-place updates and the NCHW 1x1 Conv family used throughout the recognizer;
both insert compute write-to-read barriers before their output can feed the
next graph node.

The Radeon 780M Vulkan smoke now verifies slot allocation/reuse, upload,
device binary and dependent binary-chain execution, plus device-resident
pointwise convolution against CPU FP32 reference values. The regular SIMD and
all existing Hybrid primitives remain covered. This is infrastructure for the
complete GPU-only graph, not a claim that `gpu_only` is available yet: pool,
LayerNorm, attention Softmax/QKV, transposes, concatenation, detector DB
post-processing, and remaining graph dispatches still need device paths.

### 2026-08-14: GPU-only graph operator expansion

The device arena now additionally supports a real destination-writing NCHW
suffix broadcast (`[N,C,H,W]` with compact `[C,1,1]` coefficients) and exact
Erf-form GELU on the shader. Both are numerically checked by
`ppocr_vulkan_smoke` on the AMD Radeon 780M; neither reads an intermediate
activation back through the CPU. The broadcast form is required for SE gates
and compact graph scale paths, while exact GELU covers a fused-model operator
present in every supported detector/recognizer pair.

This does **not** enable `--backend gpu`: the complete graph still lacks
device implementations for graph concatenation/shape-slice metadata handling,
unfused resize forms, SE composition, and the full OCR image/DB/CTC pipeline.
Construction continues to fail explicitly rather than silently executing an
activation on CPU. No end-to-end GPU-only benchmark is recorded until that
contract can be exercised and compared fairly.

The same arena now includes device-resident two-to-four input `Concat` and
integer-factor asymmetric/floor nearest `Resize` (including unequal height /
width factors). These are validated with multi-outer-block and `N=2` smoke
cases. They remove two more detector-FPN activation boundaries; they do not
change the strict GPU-only availability gate because a complete model graph
and OCR front/back-end still require verified device dispatches.

Medium-model SE is now represented as one device-resident arena routine:
spatial mean, both 1x1 projections, ReLU, HardSigmoid, and final channel
multiply all remain in Vulkan buffers. The Radeon smoke checks every output
against an independent FP32 reference for `N=2,C=4,reduced-C=3,plane=13`.

The ONNX executor now also has a strict Vulkan graph-execution scaffold with
device value metadata, graph-use-count slot reclamation, device copy for
branch fan-out, metadata-only reshape/squeeze/unsqueeze, pointwise unary and
binary aliases, Concat, and fused SE routing. It returns failure immediately
on an uncovered graph node; the public GPU-only gate remains closed until the
entire detector, recognizer, and OCR pipeline can use that path.

The strict graph executor now maps normal/fused NCHW Conv and Max/AveragePool
to their device-arena implementations. When an activation fan-out prevents an
in-place update, it first uses the new GPU copy primitive; this preserves
graph arithmetic rather than overwriting a still-live branch. Unsupported
geometry or nodes still terminate GPU-only execution instead of invoking CPU.

### 2026-08-15: strict Vulkan graph coverage extended

The strict device graph now also covers detector depthwise/pointwise blocks,
residual 1x1 projections, compact SE-style broadcasts, spatial means,
integer nearest resize and resize-add, 2x2 stride-two transpose convolution,
SAME padding for Conv/Pool, inference BatchNorm channel-affine, rank 2--4
transpose, and final matrix projection/Softmax. `FusedConvHardSigmoid` was
corrected to run its exported alpha/beta activation as a separate Vulkan unary
after convolution; it no longer risks returning an ungated Conv result.

`ppocr_gpu_graph_smoke` was added as a direct CPU-versus-strict-device graph
tool. It deliberately does not mark public `gpu_only` OCR as available: the
full image preprocessing, DB detector postprocessing, crop generation and
CTC decoding must still be moved onto Vulkan before that user-facing contract
can be truthfully enabled. On 2026-08-15 the Radeon 780M low-level Vulkan and
SIMD smoke suites passed; full-graph parity is still gated while the remaining
recognizer attention nodes and final numerical-error envelope are completed.

### 2026-08-15: GPU-only recognizer scheduling progress

The strict Vulkan executor now runs the PP-OCRv6 small/medium recognizer's
dynamic shape-metadata chains (`Shape`/`Slice`/`Squeeze`/`Unsqueeze`/`Concat`),
scalar attention scale, fused QKV split, rank-3/rank-4 attention `MatMul`,
Softmax, and residual transformer blocks without downloading an activation.
Only integer shape descriptors are interpreted on the host; feature maps stay
in arena buffers. The generic GEMM path is the current correctness baseline;
the experimental tiled GEMM requires `PPOCR_ENABLE_VULKAN_GEMM_TILE=1`.

Verification on the AMD Radeon 780M: `ppocr_vulkan_smoke` and
`ppocr_kernel_smoke` pass, and the normal tiny Hybrid OCR benchmark remained
functional at **10.775 ms** mean over two measured runs on `line_en.ppm`.
The ordinary-Conv shared-memory route is now gated by the same large-output
threshold in host dispatch and shader dispatch. Previously a compact SE gate
could be launched through the generic host geometry but interpreted as a
channel-tiled shader geometry, leaving stale arena values. With that mismatch
removed, strict CPU-versus-Vulkan forward parity now passes for every shipped
network on the Radeon 780M: tiny/small/medium detector at `1x3x960x960`, and
tiny/small/medium recognizer at `1x3x48x320`. The largest observed final
absolute errors are `2.42644e-06`, `6.57933e-06`, and `7.00675e-09` for the
detectors, and `5.88894e-05`, `2.09391e-04`, and `8.39829e-05` for the
recognizers, respectively.

`ppocr_gpu_graph_smoke` now accepts an optional `RUNS` argument and prints
CPU/GPU graph means, including the current GPU graph boundary. The initial
two-run reference on this UMA Radeon machine measured tiny detector
`124.284 / 610.187 ms`, tiny recognizer `8.190 / 173.200 ms`, small detector
`219.862 / 1613.830 ms`, small recognizer `20.697 / 370.361 ms`, medium
detector `1033.580 / 8530.580 ms`, and medium recognizer `33.085 / 577.311
ms` (CPU / strict Vulkan). These are correctness-baseline measurements, not
a GPU speed claim: each current graph operator submits and waits separately,
so command-batch compilation and persistent model constants are the next
Vulkan performance work.

### 2026-08-15: strict graph single-submit command recording

The strict neural Vulkan executor now records a complete forward graph into
one command buffer and waits only at its output boundary. Every dispatch gets
an immutable descriptor set from a dedicated resettable graph pool, preventing
later descriptor writes from changing earlier recorded bindings. Hybrid keeps
its original synchronous descriptor pool and full-boundary admission policy.

The local Radeon 780M verification passed the Vulkan primitive suite, SIMD
kernel suite, and all six strict graph parity cases at the published shapes.
The current two-run CPU / Vulkan means are tiny detector **223.271 / 1042.440
ms**, tiny recognizer **29.517 / 319.981 ms**, small detector **195.110 /
1249.390 ms**, small recognizer **18.378 / 291.485 ms**, medium detector
**812.492 / 10669.200 ms**, and medium recognizer **39.293 / 564.885 ms**.
Final tensor parity was exact in these runs (max absolute error `0`). The UMA
driver still makes this graph implementation slower than the AVX-512 CPU
baseline; eliminating submission fences alone is not enough to claim a
speedup, so the public GPU-only OCR contract remains closed.

### 2026-08-15: persistent GPU model weights and lower Conv tile threshold

Strict `OnnxLite` GPU models now retain their immutable initializer slots in a
model-owned Vulkan arena. Activations are released after every successful or
failed run, and the arena is guarded for the complete graph recording; this
removes repeated model-weight allocation/upload and prevents per-inference
arena growth. The ordinary-Conv shared-filter tile threshold was lowered from
16,384 to 4,096 output pixels in both host and shader dispatch logic, while
retaining the compact SE fallback path.

With a warmed model, Radeon 780M three-run strict graph means were **tiny
detector 111.399 / 264.214 ms**, **small detector 229.969 / 858.226 ms**,
**medium detector 820.237 / 8059.020 ms** (medium: two runs), **tiny
recognizer 7.915 / 86.976 ms**, **small recognizer 18.101 / 122.982 ms**, and
**medium recognizer 34.873 / 354.897 ms** (CPU / Vulkan). Exact parity held
for all six models. The tiny detector resolution ladder also passed at
`320²`, `640²`, and `960²` with zero final error; warmed Vulkan means were
**40.471**, **109.382**, and **264.214 ms**, respectively. A real `N=4`
tiny-recognizer batch at `4x3x48x320` also remained exactly equal to CPU.

These results are a substantial reduction of strict-device scheduling and
weight-transfer overhead, but this Radeon 780M UMA configuration is still not
faster than its AVX-512 CPU path for complete PP-OCRv6 graphs. Hybrid retains
its measured per-segment policy: when Vulkan is equal or faster for the full
boundary it is selected, otherwise CPU is retained to avoid consuming GPU
time without a latency benefit.

### 2026-08-15: expanded Conv tile range

The ordinary-Conv shared-weight tile is now eligible from a `256`-pixel
output plane (host and SPIR-V use the same gate), extending reuse to smaller
detector feature maps without changing compact SE convolutions. The regenerated
shader and strict graph test passed with final error `0` on tiny detector
`320x320` and `960x960`; warmed Vulkan means were **37.13 ms** and
**236.52 ms**, respectively. Lowering the threshold again to `64` regressed
both shapes, so `256` is retained.

### 2026-08-15: GPU-only front/back-end groundwork

The Vulkan arena now has a live-slot graph handoff API, fused BatchNorm+Swish
dispatch, RGB bilinear-resize+NCHW-normalization shader, device threshold-mask
shader, and device CTC Top-1 reduction. `ppocr_gpu_frontend_smoke` verifies
the RGB path on Radeon 780M; its maximum detector-normalized difference from
the scalar front-end was **0.0175072**, below one source uint8 normalization
step. The strict tiny detector and recognizer graph regressions remained exact
at their final tensors.

This is deliberately not advertised as a completed public GPU-only OCR mode:
DB box scores/unclip parity and whole-image decoded-text parity still require
an end-to-end validation suite. Accordingly `Backend::gpu_only` continues to
fail explicitly rather than returning potentially different OCR results. The
experimental device orchestration never falls back to the CPU NN executor,
but it is not enabled through the public API until that parity evidence exists.

### 2026-08-15: strict graph memory ladder and default recognizer GEMM tile

`ppocr_gpu_graph_mem_soak` is a new model-level Vulkan memory regression tool.
It warms one model, drives it through an ascending resolution/batch ladder,
then repeats a small input and reports both allocated arena capacity and live
bytes. Capacity is deliberately reusable high-water storage; the required
invariant is that live bytes return to the model's persistent-weights baseline
after every run.

On the Radeon 780M, tiny detector `320x320 -> 640x640 -> 960x960 -> 320x320`
grew capacity to **214,007,808 bytes** while live bytes returned after every
sample to **1,713,620 bytes**. Tiny recognizer
`1x3x48x80 -> 1x3x48x320 -> 4x3x48x320 -> 8x3x48x640 -> 1x3x48x80` reached
**67,633,152 bytes** and recovered each time to **4,417,112 bytes**. The
same invariant passed on small detector (**303,267,840 / 9,813,412 bytes**)
and medium recognizer (**191,889,408 / 76,456,032 bytes** capacity/live
baseline). This explicitly covers both varied detector images and recognition
batch/width growth without retaining dynamic activations after a large run.

The verified non-Swish strict-graph GEMM tile is now default-enabled for
`rows >= 8, depth >= 16, columns >= 128`; set
`PPOCR_DISABLE_VULKAN_GEMM_TILE=1` for an A/B fallback. On the same host,
final tensor parity remained exactly zero for all three recognizers. The
current short strict-recognizer means at `1x3x48x320` were tiny
**9.869 / 37.195 ms**, small **23.498 / 107.000 ms**, and medium
**45.271 / 389.620 ms** (CPU / Vulkan). This is an improvement for the
device graph but not a claim that this AVX-512 + Radeon 780M UMA system beats
CPU or ONNX Runtime end-to-end; Hybrid still uses its full-boundary
no-slower admission rule.

### 2026-08-15: parallel Softmax writeback

The strict Vulkan Softmax now retains its sequential max/sum reduction for
numerical stability, then spreads the independent exponential/writeback stage
across the 256-lane workgroup. This removes a serial device loop in
recognizer attention and terminal class Softmax without adding an activation
allocation. Final tensor parity remained **0** for tiny, small, and medium
recognizers at `1x3x48x320`, and at a real `4x3x48x320` batch. The short
post-change means were tiny **37.470 ms**, small **94.376 ms**, and medium
**382.981 ms** for the single-crop strict Vulkan graph; the small recognizer
improved from the preceding **107.000 ms** sample. Device output is still
slower than the local AVX-512 CPU baseline, so this is a verified incremental
improvement rather than an ONNX Runtime superiority claim.

The public `gpu_only` OCR switch remains deliberately disabled. Image resize
and normalization upload, DB connected-component/unclip postprocessing, crop
generation, CTC top-1/decode, end-to-end output parity, and memory recovery
must be completed before the public contract can truthfully promise complete
GPU-only OCR.

### 2026-08-15: device-resident OCR path validation gate

The experimental OCR route now feeds strict Vulkan detector/recognizer graphs
from the fused Vulkan RGB resize, BGR conversion and normalization dispatch;
it no longer builds detector or recognizer NCHW activations with CPU floating
point code. Detector probabilities are downloaded for the existing DB
connected-component/score/unclip control path, and only compact CTC Top-1
metadata is downloaded for string assembly. `ppocr_gpu_ocr_smoke` compares
the public OCR result (text, quad, bounding rectangle and confidence) with
CPU-only OCR.

On the Radeon 780M, tiny and small models passed `en.ppm`, `line_en.ppm` and
`mixed.ppm` (6 decoded regions each), with maximum confidence differences
of **0.0000061** and **0.0000019**, respectively. Tiny also passed `en.ppm`
alone at **0.0000026**. These runs establish that the NN graph has no silent
CPU operator fallback, but do not establish complete release coverage:
the dense 1920x1080 page diverged in detector/recognizer results (tiny result
72; small result 16), and medium detector strict submission at its real
`1x3x160x704` shape produced an all-zero probability map (while its 128x704
pipeline smoke passed). Therefore `full_graph_gpu_available` deliberately
remains false and `Backend::gpu_only` continues to reject construction. This
is a correctness gate, not a performance claim; no ONNX Runtime comparison or
superiority is asserted for the incomplete GPU-only route.

### 2026-08-15: GPU-only completion status

The implementation now has an all-device neural graph path, fused RGB-to-NCHW
front end, device CTC Top-1 and no CPU floating-point inference fallback.
However, it is still an experimental path rather than the requested released
GPU-only mode: on the Radeon 780M, the medium detector's true
`1x3x160x704` graph still returns `VK_ERROR_DEVICE_LOST` at the final
896-channel `5x22` depthwise block, even with one-node submissions. The
standalone arena regression for that exact depthwise geometry passes, which
isolates the remaining defect to full-graph/resource interaction. In addition,
the existing DB connected-component/unclip geometry stage remains CPU control
code, and dense `1920x1080` OCR parity is not yet exact. The public gate is
therefore intentionally still closed; advertising it as complete GPU-only
inference would be incorrect.

### 2026-08-15: GPU-only revalidation (current workspace)

The strict Vulkan implementation was rebuilt after repairing the source file
and revalidated on **AMD Radeon 780M Graphics** with
`PPOCR_GPU_ONLY_SEGMENT_NODES=1`.  It remains a no-CPU-fallback neural graph
path, but the public `Backend::gpu_only` gate is still intentionally false.
The following current CPU-versus-device graph checks passed:

| Model | Shape | Max absolute output error | CPU mean | Vulkan mean |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | `4.83182e-07` | 28.844 ms | 224.343 ms |
| small detector | `1x3x160x704` | `1.91038e-07` | 41.481 ms | 301.020 ms |
| medium recognizer | `1x3x48x320` | `8.39829e-05` | 60.583 ms | 552.145 ms |

The Vulkan primitive suite also passes, including the standalone
`C=896,H=5,W=22` depthwise convolution that resembles the medium detector
terminal block.  The remaining release blocker is reproducible only in the
complete medium detector graph: `ppocrv6_medium_det.onnx` at
`1x3x160x704` loses the Radeon device (`VK_ERROR_DEVICE_LOST`) even with one
node per command-buffer submission.  This makes public GPU-only OCR unsafe;
therefore no ONNX Runtime performance-superiority claim is made.

`ppocr_gpu_graph_mem_soak` now provides a device-arena recovery check.  On the
tiny detector, its `1x3x64x128 -> 1x3x96x256 -> 1x3x128x512 -> 1x3x64x128`
run grew reusable capacity from 18,874,368 to 27,721,728 bytes, while live
bytes returned after **every** sample to the 1,713,668-byte immutable-model
baseline.  Capacity is intentionally retained for reuse; recovered live bytes
are the leak/lifetime invariant.
## Model files

Get `ocr-models.zip` from the MaClaw `Model_Release` and unpack it.  Pass a
matching detector, recognizer, and dictionary.  The small/medium dictionary
is `D:\workprj\aicoder\corelib\ocr\dict_ppocrv6.txt`; the model archive
contains `dict_ppocrv6_tiny.txt` for the tiny recognizer.

## Demo

The included demo uses binary P6 PPM to avoid coupling a specific image
decoder to the inference library:

```powershell
.\build\ppocr_demo.exe ppocrv6_tiny_det.onnx ppocrv6_tiny_rec.onnx dict_ppocrv6_tiny.txt image.ppm
```

For PNG/JPEG/BMP, decode with the host application's existing image library
and construct `ppocr::Image { width, height, packed_rgb }` before calling
`ppocr::OCR::Recognize`.

## Scope

The C++ pipeline contains PP-OCRv6 BGR preprocessing, dynamic detection and
recognition sizing, CTC greedy decoding, and DB-style threshold/score/unclip
text region extraction.  Its ONNX reader accepts the standard embedded
float/int initializers used by the official model bundle; it intentionally
does not support arbitrary ONNX graphs, custom operators, or external tensor
data.

### 2026-08-15: strict GPU-only execution revalidation

The strict Vulkan graph executor now uses a device-resident submission boundary
per graph node by default. This is a GPU-to-GPU synchronization boundary: model
constants and live activations remain in `VulkanTensorArena`, and there is no
CPU operator fallback. A compact, separate depthwise SPIR-V pipeline is used
for narrow high-channel feature maps, avoiding the monolithic-shader tail on
Radeon UMA hardware.

On AMD Radeon 780M, CPU versus strict GPU graph checks passed for:

| Model | Shape | Max absolute output error | CPU mean | Vulkan mean |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | `4.83182e-07` | 22.184 ms | 113.042 ms |
| small detector | `1x3x160x704` | `1.91038e-07` | 42.603 ms | 193.311 ms |
| medium detector | `1x3x160x704` | `6.56222e-04` | 128.313 ms | 757.807 ms |
| medium recognizer | `1x3x48x320` | `8.39829e-05` | 39.960 ms | 472.034 ms |

The medium detector memory ladder (`64x128 → 96x256 → 128x512 → 160x704 →
64x128`) also passed: its grow-only arena reached `291,667,968` bytes, while
live allocation returned after every run to the immutable-model baseline of
`61,946,548` bytes.

Public `Backend::gpu_only` remains deliberately unavailable. Tiny and small
completed end-to-end CPU/GPU OCR parity on `en.ppm` (maximum confidence errors
`0.0000029` and `0.0000001`); however, a real-image medium detector input still
reproducibly produces `VK_ERROR_DEVICE_LOST` on this Radeon 780M at the final
896-channel depthwise layer. The same graph succeeds with the deterministic
regression tensor, therefore this is a driver/input-specific reliability
failure rather than an unsupported ONNX operator. Enabling the public API until
that real-image path is stable would violate the requested correct inference
contract. The measured Vulkan route is also slower than the AVX-512 CPU on this
integrated GPU, so no superiority claim is made.

### 2026-08-15: GPU-only arena lifetime and current recheck

Strict Vulkan graph segments now close only after their final input consumers
have been retired.  The following segment opens lazily, after the completed
fence, so released activation buffers are eligible for arena reuse immediately.
This is a device-to-device scheduling/lifetime change: no activation is
downloaded, re-uploaded, or evaluated by the CPU.

Current Radeon 780M checks with `PPOCR_GPU_ONLY_SEGMENT_NODES=1`:

| Model | Shape | Max absolute output error | CPU mean | Vulkan mean |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | `4.83182e-07` | 19.293 ms | 100.492 ms |
| small detector | `1x3x160x704` | `1.91038e-07` | 29.877 ms | 187.476 ms |
| medium detector | `1x3x128x704` | `7.48696e-09` | 85.258 ms | 716.522 ms |
| medium recognizer | `1x3x48x320` | `8.39829e-05` | 40.103 ms | 481.122 ms |

The tiny-detector device memory ladder
`64x128 -> 96x256 -> 128x512 -> 64x128` recovered to its
`1,713,668`-byte immutable-constant baseline after every sample.  Reusable
capacity grew from `18,874,368` to `27,525,120` bytes and remained stable on
the final small input.

The real medium detector at `1x3x160x704` still loses the Radeon device at
`p2o.pd_op.depthwise_conv2d.12.0` (`896x5x22`, `VK_ERROR_DEVICE_LOST`), even
with one device-resident node per fenced submission and isolated GPU copies.
Therefore `full_graph_gpu_available` remains false: enabling the public
`gpu_only` API would advertise a mode that is not correct on the tested
hardware.  These measurements also show that this strict implementation is
not yet faster than the local AVX-512 CPU, so the README makes no GPU or ONNX
Runtime superiority claim.

`PPOCR_GPU_ARENA_RETAIN_MB` is available for deployment diagnosis of the
device arena.  Its default is `128` MiB; `0` retires every completed free
activation allocation between fenced GPU segments without touching live
values or immutable model constants.  It did not resolve the Radeon medium
`160x704` driver reset, so it is a memory-pressure control rather than a
claim of GPU-only completion.

### 2026-08-15: GPU-only reliability status (latest)

The strict Vulkan graph remains fully device-resident between its input and
output boundaries: all PP-OCRv6 neural operators execute in Vulkan buffers,
and failure is reported rather than silently executing a CPU operator. The
experimental generated `vulkan_depthwise_tail.comp` route was removed because
the runtime was still bound to the established checked-in depthwise SPIR-V
module; retaining an unused second shader would make qualification results
ambiguous.

Current Radeon 780M verification with `PPOCR_GPU_ONLY_SEGMENT_NODES=1`:

| Model | Shape | Max absolute output error | CPU mean | Vulkan mean |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | `4.83182e-07` | 21.400 ms | 103.202 ms |
| small detector | `1x3x160x704` | `1.91038e-07` | 35.271 ms | 182.782 ms |
| medium detector | `1x3x128x704` | `7.32689e-09` | 103.195 ms | 779.817 ms |
| medium recognizer | `1x3x48x320` | `8.43406e-05` | 42.774 ms | 492.075 ms |

The complete medium detector at its production `1x3x160x704` shape still
reliably reports `VK_ERROR_DEVICE_LOST` at the final `C=896, H=5, W=22`
depthwise block. The standalone primitive and the `128x704` full graph pass,
so the fault is an unresolved full-prefix/driver interaction, not a CPU
fallback or missing ONNX operator. `Backend::gpu_only` therefore remains
intentionally unavailable. The measured device route is also slower than the
local AVX-512 CPU, so the project makes no performance-superiority claim over
ONNX Runtime or CPU inference.

### 2026-08-15: GPU-only qualification recheck

The strict neural-graph implementation remains pure Vulkan: activations and
model constants stay in `VulkanTensorArena`, and a failed device graph throws
instead of falling back to the CPU executor.  The temporary fresh-allocation,
arena-purge, and isolated-depthwise diagnostics used to investigate the Radeon
reset were removed because they did not resolve it; this keeps the production
path reproducible and free of behaviour-changing environment switches.

The current clean build (`cmake -S . -B build-gpu-only-clean`) and Radeon 780M
recheck passed Vulkan primitive coverage plus strict graph parity for tiny and
small detector `1x3x160x704`, medium detector `1x3x128x704`, and medium
recognizer `1x3x48x320`.  The GPU RGB front-end -> detector hand-off passed on
the real `en.ppm` page for tiny and small.  The production medium detector
`1x3x160x704` still returns `VK_ERROR_DEVICE_LOST` at its final `896x5x22`
depthwise operation.  Consequently, `Backend::gpu_only` and
`full_graph_gpu_available` remain disabled: enabling a supposedly complete
GPU-only API before that failure and end-to-end medium OCR parity are fixed
would violate the correctness contract.  No GPU/ONNX Runtime speed claim is
made for this UMA adapter.

### 2026-08-15: Vulkan full-OCR requalification and MSVC portability

### 2026-08-16: complete GPU-only reliability work — current status

The strict Vulkan executor now builds a second, purpose-specific SPIR-V module
for the medium detector's final `896x5x22` depthwise convolution.  The module
uses the same five-buffer descriptor layout and 44-byte push constants as the
main graph, but contains only the scalar pad-one depthwise operation.  Thus it
does not create a CPU fallback or an activation copy: model constants and all
intermediate tensors remain in the Vulkan arena.

The shader embedding rule was also fixed for multi-shader builds: each SPIR-V
header now uses an output-specific staging filename, preventing one embed pass
from truncating the other generated C++ header.

On the local AMD Radeon 780M / LLPC stack, the isolated tail pipeline still
ends the production medium detector `1x3x160x704` execution with
`VK_ERROR_DEVICE_LOST`.  It also fails when every node is separately fenced,
when the command buffer object is fresh for every segment, and when free
activation retention is disabled.  Tiny detector `1x3x160x704`, small
recognizer `1x3x48x320`, and low-level Vulkan primitive coverage pass after
this change.  Consequently the public `Backend::gpu_only` gate remains closed
for correctness: a complete GPU-only API must not claim support on this
adapter until the medium production model passes repeated end-to-end OCR
parity.  No CPU neural-network fallback is taken on failure.

The Vulkan source now defines `NOMINMAX` before including Windows/Vulkan
headers. This prevents the Windows `min`/`max` macros from corrupting C++
`std::min`/`std::max` calls, and a clean MSVC 19.51/Ninja Release build now
compiles the Vulkan backend and smoke tools. The low-level depthwise route
also no longer records the same descriptor-set bind twice per dispatch.

On AMD Radeon 780M, refreshed strict-graph checks (three warm passes, upload,
graph, and output download included) were:

| Model | Shape | Max absolute output error | CPU mean | Vulkan mean |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | `4.85918e-07` | 43.604 ms | 61.394 ms |
| small detector | `1x3x160x704` | `1.92900e-07` | 76.702 ms | 123.783 ms |
| medium detector | `1x3x128x704` | `7.23958e-09` | 156.387 ms | 627.358 ms |
| tiny recognizer | `1x3x48x320` | `4.70877e-06` | 12.895 ms | 54.201 ms |
| small recognizer | `1x3x48x320` | `8.94070e-07` | 26.585 ms | 153.499 ms |
| medium recognizer | `1x3x48x320` | `4.17233e-06` | 54.486 ms | 407.467 ms |

GPU RGB front-end plus strict graph validation passed for tiny and small
detectors on five varied PPM pages (`en`, `zh`, mixed, English-line and
Chinese-line), spanning `64x320`, `128x704`, and `192x704` detector inputs.
The corresponding full public OCR parity test passed all five pages for tiny
and small: 9 decoded results, with worst confidence deviation `2.9e-06` and
`1.7e-06` respectively. The ascending tiny detector device-arena regression
`64x128 -> 96x256 -> 128x512 -> 160x704 -> 64x128` reached 41,385,984 bytes
of reusable capacity and returned after every sample to the 1,713,668-byte
constants-only live baseline.

The production medium detector at `1x3x160x704` still loses the Radeon
device during the complete FPN tail. Its graph at `1x3x128x704`, standalone
operators, and the tiny/small public OCR paths pass, but the production-size
medium public OCR path does not. Therefore `gpu_only` remains hard-gated;
it never silently falls back to CPU. Hybrid keeps its per-shape full-boundary
rule and only selects Vulkan when measured upload + execution + readback is no
slower than CPU. These results do not support a claim of being faster than
ONNX Runtime on this UMA GPU.

After a device-loss return, the Vulkan runtime now tears down that invalid
logical device before its next use. A following strict tiny-graph run and the
Vulkan primitive suite both completed successfully in the same process
sequence, confirming that recovery remains GPU-only and does not turn a later
request into a CPU fallback.

### 2026-08-15: GPU-only allocator/recovery hardening

The strict Vulkan arena now exposes a monotonically changing resource
generation. `OnnxLite` compares it before each GPU-only run and discards its
model constant-slot cache after a device reset, then re-uploads constants to
the new logical device. This fixes the stale-slot ownership hazard after a
`VK_ERROR_DEVICE_LOST` recovery without allowing any CPU execution path.

Arena allocations are also rounded only to the required 64-KiB granularity
(rather than adding an unused 1.5x reserve to every independent buffer), and
the graph descriptor pool has 8,192 sets of headroom. On the Radeon 780M this
reduced the medium detector's observed allocated arena capacity before its
FPN-tail reset from about 146 MiB to about 128 MiB. The `1x3x160x704` medium
detector nevertheless still reaches a driver `VK_ERROR_DEVICE_LOST` around
the first FPN `Resize + Add` tail operation. Disabling the FPN fusion and
placing an explicit device-only submit/fence between resize and add did not
remove that fault. The public GPU-only gate remains closed until the complete
medium detector and end-to-end OCR parity pass; there is no silent CPU
fallback and no unsupported performance claim.

Post-change MSVC Release checks on AMD Radeon 780M passed the Vulkan primitive
suite, tiny detector strict graph (`1x3x160x704`, max error `4.85918e-07`,
GPU `52.2832 ms`) and small recognizer strict graph (`1x3x48x320`, max error
`8.9407e-07`, GPU `96.6479 ms`). Those GPU timings remain slower than the
local AVX-512 CPU for these shapes, so Hybrid retains its full-boundary timing
admission rule.

### 2026-08-15: GPU-only medium-tail qualification

The strict Vulkan executor now has an explicitly exercised scalar FP32
depthwise dispatch for the high-channel `896x5x22` tail shape.  It uses an
unallocated shader mode (`52`) and correctly dispatches workgroups (256
invocations each), avoiding the earlier frontend-mode collision and the
erroneous one-workgroup-per-output-value launch.  The Vulkan primitive smoke
validates this exact `C=896, H=5, W=22, 3x3, pad=1` path against a CPU
reference.  The full medium detector at `1x3x160x704` nevertheless still
returns `VK_ERROR_DEVICE_LOST` immediately after its final tail dispatch on
the local Radeon 780M/LLPC stack.  Therefore this remains a diagnostic,
GPU-only kernel improvement rather than a completed public GPU-only release:
the `gpu_only` gate stays closed and no CPU fallback or performance claim is
introduced.

### 2026-08-15: device-local Vulkan arena experiment

The Vulkan tensor arena now supports a device-local storage-buffer route with
host-visible staging buffers. `vkCmdCopyBuffer` transfers only public graph
inputs, constants, and requested outputs; all intermediate PP-OCR feature maps
remain in device-local buffers. Set `PPOCR_GPU_ARENA_DEVICE_LOCAL=0` only to
run the former directly-mapped compatibility path for comparison.

On the Radeon 780M, primitive arithmetic, image front-end parity, and the
tiny detector graph still pass with this route. The transfer-backed graph did
not resolve the production medium `1x3x160x704` device loss, and on this UMA
adapter its additional boundary copies did not improve timing. The default
CPU/Hybrid admission rule therefore remains unchanged: Hybrid selects Vulkan
only when a measured complete boundary is no slower than CPU. No ONNX Runtime
superiority is claimed from this experiment.

### 2026-08-15: multi-size GPU graph and residency recheck

The graph smoke matrix now covers detector inputs from `64x128` through
`192x704`, plus recognizer widths `64` and `320`, across all tiny/small/medium
families. On the Radeon 780M, all listed runs had device/CPU output parity:

| Model | Shape | CPU ms | Vulkan ms | Max abs. error |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x64x128` | 5.675 | 16.897 | `1.13e-09` |
| tiny detector | `1x3x192x704` | 22.467 | 55.239 | `2.66e-05` |
| small detector | `1x3x64x128` | 10.150 | 38.459 | `7.74e-09` |
| small detector | `1x3x192x704` | 41.085 | 99.252 | `4.60e-07` |
| medium detector | `1x3x64x128` | 24.039 | 75.144 | `4.22e-10` |
| medium detector | `1x3x128x704` | 132.380 | 477.800 | `7.24e-09` |
| tiny recognizer | `1x3x48x64` | 5.274 | 16.755 | `3.52e-06` |
| small recognizer | `1x3x48x320` | 20.938 | 82.170 | `8.94e-07` |
| medium recognizer | `1x3x48x320` | 46.774 | 351.068 | `4.17e-06` |

The ascending tiny detector memory soak
`64x128 -> 96x256 -> 128x512 -> 160x704 -> 64x128` ended every sample at the
same constants-only live baseline (`1,713,668` bytes), proving activation
release after the largest input. `ppocr_gpu_graph_mem_soak` now reports both
logical device capacity and total allocated residency; at the largest shape
these were `31,719,424` and `63,438,848` bytes respectively because the
device-local route has a matching host staging allocation. These measurements
remain slower than the local AVX-512 CPU and are not ONNX Runtime comparisons;
Hybrid consequently admits only independently measured no-slower segments.

### 2026-08-16: GPU-only full-path requalification

The strict Vulkan path remains the intended implementation for complete
GPU-only OCR: image resize/normalization, detector and recognizer graphs, and
terminal CTC Top-1 execute on Vulkan storage. Only compact OCR control data is
returned to the host for DB connected-component traversal and UTF-8 assembly;
there is no CPU neural-network fallback. `gpu_only` remains hard-gated until
the production detector and end-to-end OCR suite pass.

For the Radeon 780M/LLPC failure investigation, the scalar `896x5x22`
depthwise tail can now be split into GPU-resident channel windows. The normal
path uses four windows; `PPOCR_GPU_TAIL_SLICES=1..896` is a diagnostic
granularity setting, `PPOCR_GPU_TAIL_SUBMIT_SLICES=1` fences those windows
without copying activations to the host, and `PPOCR_GPU_TAIL_GENERIC=1`
selects the original packed route. All three routes still report
`VK_ERROR_DEVICE_LOST` for the production medium detector
`1x3x160x704`; with per-window submission the loss occurs on the first
`C=0..223` scalar window. This localizes the remaining issue to the driver
interaction before/at that dispatch rather than an unsupported ONNX operator.

Current release checks passed: Vulkan primitive coverage, tiny and small
detector graphs at `1x3x160x704`, medium detector at `1x3x128x704`, and medium
recognizer at `1x3x48x320`. Measured one-run timings on this shared host were
`24.018/50.106 ms`, `37.939/86.509 ms`, `121.056/487.819 ms`, and
`51.734/358.526 ms` (CPU/Vulkan); maximum output errors were respectively
`4.86e-07`, `1.93e-07`, `7.24e-09`, and `4.17e-06`. The repeated ascending
memory soak also passed, returning every sample to the `1,713,668`-byte
constants-only live baseline; at `160x704`, logical capacity was
`31,719,424` bytes and device-plus-staging residency was `63,438,848` bytes.
These results are slower than the local AVX-512 CPU and are not an ONNX Runtime
comparison, so no unsupported GPU performance claim is made.

### 2026-08-16: pointwise-GELU Vulkan fusion qualification

The Vulkan shader now has a direct pointwise-convolution plus FP32-GELU
write mode. It removes one full feature-map read/write and one graph dispatch
for `FusedDepthwisePointwiseConvGelu`; the normal mode preserves the existing
separate Vulkan unary GELU path. `PPOCR_GPU_FUSE_POINTWISE_GELU=1` selects the
new all-device fusion for adapter qualification, with identical graph-output
parity (tiny `4.86e-07`, small `1.93e-07`, medium `7.24e-09`).

On this AMD Radeon 780M/LLPC system, ten-run detector comparisons showed the
fusion is device-dependent: tiny `35.466 -> 33.748 ms` (5.1% faster), small
`88.883 -> 91.514 ms` (3.0% slower), and medium `546.302 -> 643.987 ms`
(17.9% slower), normal/fused. The default therefore retains the faster
separate device kernels while the fused Vulkan implementation stays available
for discrete GPUs or drivers where it wins. This preserves the Hybrid policy:
Vulkan is selected only after a full-boundary no-slower check; no CPU fallback
or unsupported performance claim was introduced.

### 2026-08-16: complete GPU-only OCR qualification

`Backend::gpu_only` now executes the complete PP-OCRv6 neural path on Vulkan:
RGB resize/normalization, detector, recognizer, and CTC Top-1 remain on device.
Only DB connected-component control data and final UTF-8 text assembly return to
the CPU; there is no CPU neural-network fallback.  The shared Vulkan arena now
has an explicit detector-to-recognizer hand-off that releases idle model
constants and their backing 64 MiB suballocation blocks before the next model
is uploaded.  This prevents the two model families from being resident
together on UMA GPUs.

The matching small/medium dictionary is
`D:\workprj\aicoder\corelib\ocr\dict_ppocrv6.txt` (18,708 lines; 18,710
CTC classes); tiny uses `dict_ppocrv6_tiny.txt` (6,904 lines; 6,906 classes).
With the matching dictionaries, the public CPU-versus-GPU OCR regression passed
five varied PPM pages (`en`, `line_en`, `line_zh`, `mixed`, `zh`) for every
model tier: 9 decoded results each, with text and geometry identical.  Maximum
absolute confidence differences were `0.0494437` (tiny), `0.0136318` (small),
and `0.0081339` (medium). GPU-only uses a right-zero-padded 64-column
recognizer-width grid to avoid driver-sensitive dynamic transformer widths;
this can move its confidence aggregate but does not change decoded text or
geometry on the qualified corpus.

On the local AMD Radeon 780M / LLPC system, warmed strict-graph comparisons
(two runs; input/output boundary included) were:

| Model | Shape | CPU mean | Vulkan mean | Max abs. error |
|---|---:|---:|---:|---:|
| tiny detector | `1x3x160x704` | 22.356 ms | 45.910 ms | `4.85918e-07` |
| small detector | `1x3x160x704` | 35.894 ms | 99.619 ms | `1.92900e-07` |
| medium detector | `1x3x160x704` | 160.500 ms | 604.553 ms | `1.81899e-08` |
| tiny recognizer | `1x3x48x320` | 8.852 ms | 36.383 ms | `4.70877e-06` |
| small recognizer | `1x3x48x320` | 20.985 ms | 93.526 ms | `8.94070e-07` |
| medium recognizer | `1x3x48x320` | 51.310 ms | 370.130 ms | `4.17233e-06` |

The GPU memory soak `64x256 -> 96x512 -> 160x704 -> 64x256` passed. At the
largest tiny-detector input, arena capacity grew to `31,457,280` bytes, while
live storage returned after each image—including the final smaller image—to
the constants-only baseline of `1,713,668` bytes. These Radeon measurements
are slower than the local AVX-512 CPU and are not ONNX Runtime comparisons;
they do not support a claim of GPU superiority. Hybrid retains its
transfer-inclusive no-slower admission policy.

### 2026-08-16: GPU-only true batch scheduling

`RecognizeBatch` now has a GPU-only batch path. Pages with identical source
and detector geometry share one Vulkan RGB-resize dispatch and one detector
NCHW graph; detected crops with the same Vulkan-qualified recognition width
share one recognizer/CTC NCHW graph. The implementation retains distinct
source geometries as separate buckets, so it never pads an unlike page or
changes detector resize semantics. If a driver rejects a strict detector batch
(the local medium `N>1` high-channel tail does), it retries those pages as
independent GPU-only graphs after device recovery—never on CPU.

On four identical `720x152` `en.ppm` pages with `det_batch=4` and
`rec_batch=4`, CPU-versus-batch result parity passed. GPU-only timing on the
Radeon 780M was `434.004 ms -> 292.536 ms` serial/batched for tiny, a `1.484x`
throughput gain. The preceding small-model run measured `1480.410 ms ->
1249.591 ms` (`1.185x`). Medium safely uses the strict N=1 fallback on this
driver; the architecture-specific limitation is explicit rather than a hidden
CPU fallback. The five-image tiny/small/medium GPU-only OCR parity suite and
the ascending memory-recovery soak continue to pass after this scheduling
change.

### 2026-08-16: GPU-only single-page crop batching

The same Vulkan NCHW recognizer scheduler is now also used for repeated-width
text crops within a single `Recognize()` call. It groups only crops with the
same qualified 64-column grid, writes each crop directly into its batch plane,
then runs one strict recognizer plus device CTC Top-1 graph. Thus this is a
GPU-only throughput improvement, not a CPU recognizer fallback or a change to
the page geometry contract.

After rebuilding `build-gpu-only-final`, the five-page public OCR regression
still passed for tiny/small/medium respectively: 9 decoded results with
identical text and boxes, and maximum confidence deltas `0.0494437`,
`0.0136318`, and `0.0081339`. The GPU graph memory recovery sweep also passed
after `64x256 -> 96x512 -> 160x704 -> 64x256`: capacity reached `31,457,280`
bytes at the largest input while live storage returned after every run to its
`1,713,668`-byte constants baseline.

On the local Radeon 780M, a renewed warmed four-page `en.ppm` batch check
with `det_batch=4`, `rec_batch=4` measured tiny `530.291 -> 311.165 ms`
(`1.704x`) and small `1496.264 -> 1277.662 ms` (`1.171x`), serial/batched.
Both preserve exact GPU-only serial/batch output parity. These are
machine-specific Vulkan figures, not an ONNX Runtime comparison or a claim of
universal GPU superiority.

### 2026-08-16: GPU-only dense-page upload reuse and memory ladder

GPU-only recognizer batching now uploads each source page once per
same-width crop batch, rather than uploading the full RGB page once for every
text line. The device resize still writes each crop directly to its own NCHW
plane and the recognizer/CTC graph remains strict Vulkan. This lowers upload
traffic and peak source staging lifetime on dense pages without changing text,
box, or confidence semantics.

The cross-page scheduler also remembers a driver-rejected exact detector
batch shape. On a later occurrence it immediately schedules independent
strict GPU-only graphs instead of repeatedly provoking the same Vulkan device
recovery. The cache is keyed by source size, detector size, and N; it neither
widens to unqualified shapes nor permits CPU inference.

`bench/memory_ladder_regression.py` now accepts `--backend gpu`, so the
existing diverse `16x16` through `7680x4320` ladder can check GPU-only host
memory as well. A current tiny GPU-only one-cycle run passed with the required
post-large decline. Its final `64x64` sample was `342,462,464` private bytes
and `363,364,352` working-set bytes, versus the 5K/8K pre-high-water
`490,450,944` and `521,076,736` bytes (about 30.2%/30.3% lower). This observes
process accounting; the separate Vulkan arena regression still proves live
tensor storage returns to the constants-only baseline.

After the upload-reuse change, the five-page public GPU-only OCR parity suite
again passed for tiny/small/medium (9 results per tier; identical text and
geometry). A renewed warmed four-page batch run on the Radeon 780M measured
tiny `447.936 -> 295.321 ms` (`1.517x`) and small
`1542.215 -> 1280.809 ms` (`1.204x`), serial/batched. These are local Vulkan
measurements, not an ONNX Runtime comparison or a universal speed claim.

### 2026-08-16: measured Vulkan command-segment tuning

The strict Vulkan executor now defaults to 20 graph nodes per submitted
command segment (`PPOCR_GPU_ONLY_SEGMENT_NODES` remains an override). The
larger safe segment reduces queue/fence crossings while retaining the explicit
medium-tail device boundary and arena-lifetime hand-off. On the local Radeon
780M mixed three-line page, a 5/10/20/40-node sweep selected 20 as the best
common setting: tiny was `120.753 ms`, small `466.093 ms`, and medium
`2427.856 ms`; 40 nodes regressed all three and 32 regressed medium. The
five-page tiny/small/medium GPU-only OCR parity test still passed after making
the default change.

For context, a current local mixed-page comparison with that default reported
CPU `41.735 / 99.223 / 336.651 ms`, hybrid `52.127 / 124.219 / 531.973 ms`,
and strict GPU-only `122.299 / 479.389 / 2462.806 ms` for tiny/small/medium.
Hybrid's individual Vulkan admissions remain transfer-inclusive and select GPU
only when that measured primitive boundary is no slower than CPU; these
end-to-end figures show that this UMA adapter does not currently justify a
broader GPU claim. They are not an ONNX Runtime comparison.

### 2026-08-16: SIMD GPU-upload boundary coverage

The GPU-only RGB upload boundary now uses a runtime-dispatched byte-to-FP32
conversion: AVX-512 when the OS/CPU state enables it, AVX2 on compatible x86,
NEON on ARM, and scalar otherwise. This preserves the existing Vulkan shader
ABI and performs no CPU image resize, normalization, or neural operator—the
Vulkan front end still owns all of those operations. The change specifically
removes scalar byte-conversion pressure before GPU submission for large source
pages and detector/recognizer batches.

The kernel smoke now validates vector blocks plus an odd tail for that upload
conversion. Release validation on the Radeon 780M passed kernel smoke, the
five-page public GPU-only OCR parity suite for tiny/small/medium (same 9
decoded outputs and prior maximum confidence deltas `0.0494437`, `0.0136318`,
and `0.0081339`), and the strict graph live-storage recovery sweep
`64x256 -> 96x512 -> 160x704 -> 64x256`. The final test retained a
`1,713,668`-byte constants-only live baseline; capacity was `41,615,360`
bytes after the largest shape, which is reusable allocation capacity rather
than live activation storage.

The one-cycle tiny GPU-only process-memory ladder across the existing
`16x16` through `7680x4320` corpus passed the post-large decline gate. Its
final `64x64` checkpoint was `346,824,704` private bytes and
`365,412,352` working-set bytes, below the observed `5K/8K` high-water of
`520,888,320` and `549,412,864` bytes. This is a memory-lifetime regression,
not a claim that Windows must immediately decommit every reusable allocator
page.

The broadening corpus also exposed driver/model limits that remain explicit:
tiny/small/medium GPU-only parity passes on `en.ppm` and `mixed.ppm`, while
this Radeon/LLPC stack does not yet execute the medium strict GPU graph for
the extreme `3000x80` strip; it fails closed rather than invoking CPU neural
inference. An unrelated synthetic `ui_dense_1920x1080` page can produce
near-threshold detector-component differences between CPU and FP32 Vulkan
reductions, so it is retained as a stress case, not mislabeled as current
strict geometry parity. Consequently this update makes no claim of universal
GPU superiority or of being ahead of ONNX Runtime; that requires fresh,
matched end-to-end comparisons for every model and corpus case.

### 2026-08-16: strict GPU-only extreme-short detector qualification

The dedicated Vulkan depthwise-tail module now also qualifies the medium
detector's named terminal `896x1x30` shape, reached by the `3000x80` strip
after detector resizing to `32x960`. Its padding is decoded from the existing
Vulkan push constants, so both the original `896x5x22` route and the new
short-strip route remain a single purpose-built, device-resident FP32
dispatch. No ONNX Runtime or CPU neural fallback is introduced.

On the local Radeon 780M / AMD LLPC stack, the strict detector graph for
`1x3x32x960` completed with maximum CPU/Vulkan output error
`9.60426e-10`. The complete public GPU-only OCR check for medium on
`wide_strip_3000x80.ppm` also passed (one result; confidence delta
`0.0046322`). The standard tiny/small/medium GPU-only OCR corpus was rerun
afterwards and still passed with exact decoded text and geometry; confidence
deltas were `0.0494437`, `0.0136318`, and `0.0081339` respectively.

The extension deliberately remains narrowly qualified. The same LLPC driver
still loses the device at the medium terminal tail for a larger `17x30` map
(for example a `544x960` detector input, which appears in the 8K memory
ladder). GPU-only continues to fail closed for that unsupported shape. The
medium GPU memory-ladder report is therefore intentionally not claimed as
passing; the already-passing tiny ladder remains the current broad
small-to-large-to-small process-memory evidence. This is correctness and
coverage work, not an ONNX Runtime performance comparison or a universal GPU
speed claim.

The SPIR-V header generator was also changed to append sixteen bytes per
CMake write rather than one byte per write. This preserves the generated
header layout while reducing generation time for the 170 KiB graph shader and
prevents a timed-out build from leaving a partially emitted header for the C++
compiler to consume.

### 2026-08-16: strict GPU-only CTC confidence correction

The terminal GPU CTC Top-1 reduction now returns the selected value from the
model's already-computed Softmax tensor.  An experimental second Softmax over
those probabilities was removed because it changes the public CTC confidence
and is not a valid operator fusion.  The recognizer remains entirely
device-resident through resize/normalization, every neural-network operator,
terminal Softmax, and CTC Top-1; only the compact index/probability records
cross back to assemble public text results.

After a clean shader rebuild, `ppocr_kernel_smoke` passed and strict Vulkan OCR
regression passed for tiny and small on `en.ppm`, `line_en.ppm`, and
`mixed.ppm`, with maximum CPU/GPU confidence deltas `0.0494437` and
`0.0136318`. Medium passed the same three images plus
`wide_strip_3000x80.ppm` with a maximum confidence delta of `0.0081339`.
These runs execute no CPU neural-network operator or ONNX Runtime fallback.
As before, detector DB component traversal and final result-string assembly
are deliberately CPU control-plane work after a GPU result boundary; they are
not model inference.

### 2026-08-16: terminal Softmax-to-CTC Vulkan fusion and broad memory check

For the strict GPU-only recognizer, the terminal `Softmax -> CTC Top-1`
sequence is now fused without materialising the full terminal probability
tensor. The GPU CTC dispatch receives the final logits, finds the same
per-row maximum used by stable Softmax, and calculates only the winning
probability as `1 / sum(exp(logit - max))`. This retains greedy CTC indices
and confidence semantics while removing the terminal Softmax read/write and
dispatch. Ordinary public GPU graph output still executes its explicit
Softmax, so the fusion is restricted to the device CTC endpoint.

The three-tier GPU-only OCR regression was rebuilt after the fusion. Tiny and
small passed `en.ppm`, `line_en.ppm`, and `mixed.ppm`; medium passed those
three plus the `3000x80` short strip. Decoded text and geometry stayed exact,
with maximum CPU/GPU confidence deltas `0.0494437`, `0.0136318`, and
`0.0081339`. `ppocr_kernel_smoke` also passed.

The reproducible one-cycle tiny GPU-only ladder (`16x16` through `7680x4320`
with wide/tall recovery branches) passed the post-large-memory-decline gate.
The durable final `64x64` checkpoint was `346,030,080` private bytes and
`344,858,624` working-set bytes, below the observed pre-return high water of
`545,927,168` and `554,819,584` bytes. This demonstrates bounded host and
Vulkan arena recovery; it is not an ONNX Runtime performance claim.

The local Radeon 780M / LLPC timing evidence still does not justify claiming
that GPU-only is universally faster than ONNX Runtime or the host AVX-512
CPU. In particular, the available ONNX Runtime installation exposes only
`CPUExecutionProvider`, so no matched Vulkan/GPU-provider comparison is
available on this machine. Hybrid continues to use its transfer-inclusive
per-operator admission rule and selects Vulkan only when it measures no
slower than CPU.

### 2026-08-16: reproducible native-build ORT comparison

`bench/compare.ps1` and `bench/compare_models.ps1` now accept `-NativeExe`.
This makes a benchmark name the exact CMake build under test rather than
silently selecting `build/ppocr_bench.exe`; for example, the qualified Vulkan
release binary can be passed as
`build-gpu-only-final/ppocr_bench.exe`. The path is emitted in the JSON report
for auditability.

On the local AVX-512 host, using 16 native/ORT CPU threads, two warmups and
eight timed end-to-end runs of the same `mixed.ppm` page, the selected native
release produced the following CPU-only results against ONNX Runtime 1.28.0
`CPUExecutionProvider`:

| PP-OCRv6 model | native mean | ORT mean | native latency improvement |
| --- | ---: | ---: | ---: |
| tiny | 58.679 ms | 164.644 ms | 64.36% |
| small | 100.198 ms | 251.448 ms | 60.15% |
| medium | 433.107 ms | 760.844 ms | 43.08% |

This is a matched local CPU comparison: detector preprocessing, DB postprocess,
recognizer preprocessing and CTC decode are included by both harnesses. It
proves a substantial benefit for all three model sizes on this corpus; it
does not imply an architecture-independent guarantee. The benchmark's
installed ORT package reports only `CPUExecutionProvider` (plus Azure
metadata), so it is not evidence about an ORT GPU provider. GPU-only and
Hybrid remain available for Vulkan acceleration, while Hybrid retains its
measured no-slower-than-CPU admission policy.

### 2026-08-16: adaptive Vulkan CTC reduction experiment

The Vulkan CTC Top-1 shader now contains a cooperative 256-lane vocabulary
reduction for adapters where the terminal recognizer rows are a GPU bottleneck.
It preserves earliest-index tie breaking and computes the selected stable
Softmax probability on device. It is deliberately opt-in via
`PPOCR_GPU_CTC_COOPERATIVE=1`: the Radeon 780M / LLPC default remains the
previous scalar-per-row reduction because the driver-qualified path is faster
and materially more stable there. This keeps the advanced instruction/workgroup
path available for discrete GPUs without regressing the qualified UMA default.

The default Vulkan path was rebuilt and passed the kernel smoke plus public
GPU-only OCR parity for tiny, small and medium (medium including
`wide_strip_3000x80.ppm`). The cooperative path is therefore an explicit
deployment tuning option, not a benchmark claim until it has been measured on
the destination GPU.

### 2026-08-16: GPU-only full-network contract revalidation

The public `Backend::gpu_only` route remains a strict Vulkan neural-network
execution path: RGB resize/normalization, detector, recognizer transformer,
terminal Softmax/CTC Top-1, and recognizer crop batching execute on Vulkan;
construction or execution fails closed if the complete model graph cannot be
lowered. CPU retains only application boundaries: image-byte upload, DB output
geometry/control processing after the detector result is returned, and final
UTF-8 string assembly. No ONNX Runtime library or CPU neural operator is used
by this route.

The rebuilt release passed `ppocr_kernel_smoke`, the Vulkan RGB/threshold
front-end smoke, and CPU-vs-GPU public OCR parity: tiny and small each passed
`en.ppm`, `line_en.ppm`, and `mixed.ppm`; medium passed those plus
`wide_strip_3000x80.ppm`. Maximum confidence deltas were respectively
`0.0494437`, `0.0136318`, and `0.0081339`, while decoded text and geometry
were exact.

A fresh local Radeon 780M / LLPC steady-state tiny GPU-only probe on
`mixed.ppm` (one warm-up, three timed runs) measured `137.016 ms` mean,
`131.292 ms` median, and `7.298 FPS`. This is a device-specific Vulkan result,
not a comparison with an ONNX Runtime GPU provider. The accompanying
two-identical-page GPU batch smoke (`det_batch=2`, `rec_batch=2`, one warm-up,
two runs) preserved serial result parity and measured `221.428 ms` serial vs
`194.733 ms` batched mean (`1.137x`).

### 2026-08-16: GPU-only small-model immutable-residency cache

GPU-only OCR now keeps a detector/recognizer pair's fused immutable Vulkan
weights resident across the detector-to-recognizer hand-off when their combined
payload is at most `64 MiB` (override with `PPOCR_GPU_CONSTANT_CACHE_MB` before
constructing `OCR`). The hand-off still releases every completed activation;
only model constants remain. Larger pairs, including the local medium model,
retain the prior conservative release-and-reclaim policy for UMA safety.

This changes no inference operator or fallback policy. A cold comparison with
`PPOCR_GPU_CONSTANT_CACHE_MB=0` confirmed the intended boundary: on Radeon
780M/LLPC, tiny `mixed.ppm` with two warm-ups/eight runs changed from
`318.372 ms` (re-upload per stage) to `133.153 ms` with the resident pair
(`2.39x`). The resident run retained `128.241 ms` median and `7.510 FPS`.
Small and medium remain strict GPU-only graph executions and passed their
public OCR parity suites alongside tiny; all three confidence deltas remain
`0.0494437`, `0.0136318`, and `0.0081339` respectively.

The policy is deliberately bounded rather than a blanket cache: a tiny
GPU-only `16x16 -> 1920x1080 -> 64x64` process-memory check completed with
completed-activation memory reclaimed before the small return image. The
resident constants establish a stable service baseline; the large page raised
private bytes by `15,728,640` bytes and the final small page did not grow it
further. A full ascending-to-8K ladder remains the stronger regression gate.

### 2026-08-16: cached GPU-only 8K memory ladder and three-mode snapshot

The full tiny GPU-only memory regression was rerun after enabling bounded
immutable-pair residency, using the existing `16x16` through `7680x4320`
resolution/aspect-ratio ladder and its recovery tail. It passed
`--require-post-large-decline`. The stable warmed baseline was `250,720,256`
private bytes; the pre-return high-water was `502,628,352` private bytes /
`528,109,568` working-set bytes, while the final `64x64` checkpoint returned
to `256,557,056` / `267,509,760` bytes. The durable report is
`build-gpu-only-final/tiny_gpu_memory_ladder_constants_cached.json`.

For a separate steady-state `mixed.ppm` snapshot (16 native worker threads,
two warm-ups, eight timed runs), the same release measured:

| model | CPU-only mean | Hybrid mean | GPU-only mean | preferred local mode |
| --- | ---: | ---: | ---: | --- |
| tiny | 45.476 ms | 53.561 ms | 133.153 ms | CPU-only |
| small | 93.684 ms | 170.108 ms | 462.873 ms | CPU-only |
| medium | 376.173 ms | 542.679 ms | 2435.894 ms | CPU-only |

These figures are intentionally not presented as a universal GPU win: the
Radeon 780M is an UMA adapter and this full page is CPU-SIMD-favourable.
Hybrid's Vulkan admission remains strict and transfer-inclusive; every
qualified shape chooses Vulkan when measured GPU time is equal to or less than
the CPU segment time, otherwise it preserves the faster CPU segment and avoids
adding latency. GPU-only is available for applications that require its
all-Vulkan neural execution contract, while CPU-only remains the measured
throughput choice for this local corpus.

### 2026-08-16: GPU-only contract recheck

The release build was rechecked after the GPU-only integration pass.  The
kernel suite, Vulkan RGB resize/normalization smoke, and the real
front-end-to-detector device-slot regression all passed.  On `en.ppm`, public
CPU-versus-GPU-only OCR parity passed for tiny, small, and medium, with
maximum CTC-confidence differences of `0.0494437`, `0.0123513`, and
`0.0081339`; decoded text and box geometry were exact.  A true two-page tiny
GPU batch (`det_batch=2`, `rec_batch=2`) also completed with serial-equivalent
results, proving that the public batch scheduler submits an NCHW GPU-only
detector and batched recognizer rather than invoking a CPU neural fallback.

The term *GPU-only* is constrained precisely: packed image bytes are uploaded
from the application, and CPU performs DB connected-component geometry plus
final UTF-8 assembly after the detector/CTC result boundaries.  RGB resize and
normalization, all detector and recognizer neural operators, transformer
attention, terminal Softmax/CTC Top-1, and the device crop batches remain on
Vulkan.  No ONNX Runtime or CPU neural operator is linked or used by that
path; failure to submit any required Vulkan graph remains fail-closed.

One deliberately unhidden coverage gap remains: the current Radeon 780M/LLPC
FP32 detector run differs enough from the CPU reference around threshold
boundaries that the dense `ui_dense_1920x1080.ppm` page does not yet preserve
all CPU box/text records.  This is not a CPU fallback: it is a numerical-parity
issue to resolve before claiming broad dense-page GPU-only output equivalence.
The qualified normal-page and batch contract above remains passing.

### 2026-08-16: GPU-only DB postprocess completion

`Backend::gpu_only` now retains the detector probability map on Vulkan through
the full DB postprocess: thresholding, 8-connected-component traversal, mean
score filtering, rectangular unclip, source-coordinate conversion, and reading
order.  The device kernel uses a bounded on-device FIFO (linear in detector-map
pixels); it downloads only final `{x0,y0,x1,y1,score}` records.  The previous
full FP32 probability-map download and CPU DB component pass have been removed
from the GPU-only single-page and true-N detector-batch paths.  As before, UTF-8
assembly after the compact device CTC result is host-side application output
formatting, not neural inference.

Rebuilt `build-gpu-only-final` validation on AMD Radeon 780M/LLPC passed:

| Check | Coverage | Result |
| --- | --- | --- |
| Kernel + Vulkan front-end | SIMD kernels; RGB resize/normalize; GPU threshold | passed |
| GPU-only OCR, tiny | `en`, `line_en`, `mixed` | 6 results, max confidence delta `0.0494437` |
| GPU-only OCR, small | `en`, `line_en`, `mixed` | 6 results, max confidence delta `0.0136318` |
| GPU-only OCR, medium | `en` | 2 results, max confidence delta `0.0081339` |
| True GPU detector/recognizer batch | 2 identical tiny `en` pages, `det_batch=2`, `rec_batch=2` | serial-equivalent results; `209.975 -> 193.990 ms` (`1.082x`) |

This is a real GPU-only execution contract—no ONNX Runtime and no CPU neural or
DB floating-point fallback are linked into the route.  The published confidence
and dense-page parity caveats still apply; the table is correctness evidence,
not a claim that this UMA GPU is universally faster than CPU or an ORT GPU
provider.

The GPU-only service path also now reclaims completed transient Vulkan storage
after each public request while retaining the bounded immutable model cache.
`PPOCR_GPU_ONLY_RECLAIM_TRANSIENT=0` is available only for deployment-local
throughput A/B testing.  The new one-cycle, ascending `16x16` through
`7680x4320` memory ladder passed `--require-post-large-decline`; its durable
report is `build-gpu-only-final/tiny_gpu_memory_ladder_device_db_reclaimed_20260816.json`.
On that run, final `64x64` private bytes were `409,698,304`, below the warmed
baseline `410,071,040`, and below the pre-return peak `515,739,648`.

### 2026-08-16: native-byte GPU-only image ingress

The strict Vulkan front end now uploads the application RGB buffer as packed
native bytes.  The resize shader unpacks those bytes on device before its
existing bilinear sampling and BGR normalization; the former CPU
AVX-512/AVX2/NEON byte-to-FP32 widening step is no longer on the GPU-only
request path.  The arena remains FP32-sized for common allocator/lifetime
handling, but image ingress consumes only its byte prefix.  This is a boundary
change only: detector/recognizer graph operations, DB postprocess, and terminal
CTC reduction remain Vulkan-only, with no ONNX Runtime dependency or CPU
neural fallback.

Rebuilt validation on the local Radeon 780M/LLPC passed the kernel suite and
front-end smoke, then public strict OCR parity for tiny and small on `en`,
`line_en`, and `mixed` (six records each), and medium on the same three pages
(six records, maximum CPU/GPU confidence difference `0.0081339`). A true
two-page tiny GPU batch (`det_batch=2`, `rec_batch=2`) remained
serial-equivalent and measured `311.699 -> 227.434 ms` mean (`1.371x`) in a
three-run local probe. These are machine-specific Vulkan measurements, not a
claim about an ONNX Runtime GPU provider.

The full native-byte one-cycle GPU-only memory ladder again passed the required
post-large decline gate. The report is
`build-gpu-only-final/tiny_gpu_memory_ladder_native_rgb_upload_20260816.json`:
the warmed baseline was `409,759,744` bytes, final `64x64` was `409,784,320`
bytes, and the pre-return private-byte peak was `592,015,360` bytes. Thus the
completed large-image workspace fell by 30.8% before the final small-image
checkpoint.

### 2026-08-22: strict GPU-only requalification and CTC metadata correction

`Backend::gpu_only` remains a fail-closed Vulkan path: RGB upload/resize and
normalization, detector, Vulkan DB postprocess, crop resize, recognizer, and
CTC Top-1 run on the device. The host receives only final DB records and
compact CTC metadata for UTF-8 result assembly. No ONNX Runtime is linked, and
a Vulkan error is reported instead of selecting CPU neural inference.

The CTC hand-off now restores the CPU-visible convention that blank and
repeated rows carry zero probability. This changes only compact output
metadata after the Vulkan reduction; it neither downloads logits nor evaluates
an inference operator on CPU.

Fresh Radeon 780M / LLPC checks from `build-gpu-only-ep`:

| Check | Result |
| --- | --- |
| Tiny detector `1x3x160x704`, 2 runs | max error `5.63276e-7`; CPU/GPU mean `10.269 / 45.892 ms` |
| Small detector `1x3x160x704`, 2 runs | max error `3.05939e-7`; CPU/GPU mean `19.456 / 65.210 ms` |
| Medium detector `1x3x160x640`, 2 runs | max error `7.56991e-8`; CPU/GPU mean `80.544 / 266.526 ms` |
| Tiny GPU-only OCR corpus | 6 images × 2 runs, 63 records; max confidence delta `0.0011662` |
| Small GPU-only OCR corpus | 6 images × 2 runs, 63 records; max confidence delta `0.0002265` |
| Tiny true GPU batch (`en` × 2, `det_batch=2`, `rec_batch=2`) | serial-equivalent; `92.841 -> 83.830 ms` (`1.107x`) |

The corpus covers `en`, `mixed`, low-contrast, ultra-wide, ultra-tall, and
`1920x1080` screenshot pages. The dense synthetic UI page remains an explicit
parity stress case: on this driver its GPU DB contour can differ from the CPU
reference by a subpixel amount that becomes a different tiny recognizer crop.
It therefore remains outside the qualified strict-OCR corpus until detector
contour parity is repaired; this is a correctness gate, never a CPU fallback.
These local UMA measurements do not claim to beat ONNX Runtime or CPU SIMD.

### 2026-08-22: GPU-only execution and memory recheck

The current public `Backend::gpu_only` route is an all-Vulkan inference
contract: RGB upload/resize/normalization, detector, DB threshold/component
postprocess, recognizer crop resize, recognizer, and CTC Top-1 stay on the
device. The host receives only compact final box/CTC metadata to construct
the public UTF-8 result. A required Vulkan failure is fail-closed; it never
retries the neural graph or DB stage on CPU.

After rebuilding `build-gpu-only-ep`, the following local Radeon 780M / LLPC
checks passed:

| Check | Result |
| --- | --- |
| Kernel SIMD/Vulkan smoke | passed |
| Tiny detector `1x3x160x704` | max error `5.63276e-7`; CPU/GPU `10.816 / 44.449 ms` |
| Small detector `1x3x160x704` | max error `3.05939e-7`; CPU/GPU `40.614 / 78.777 ms` |
| Medium detector `1x3x160x640` | max error `7.56991e-8`; CPU/GPU `86.750 / 277.625 ms` |
| Tiny GPU-only OCR corpus | 6 images × 2; 63 records; exact text/geometry; max confidence delta `0.0011662` |
| Small GPU-only OCR corpus | 6 images × 2; 63 records; exact text/geometry; max confidence delta `0.0002265` |
| Tiny true device batch (`en` × 2, detector/recognizer batch 2) | serial-equivalent; `93.460 -> 85.286 ms` (`1.096x`) |

The same tiny instance was then warmed over an ascending source ladder
(`16x16`, `en`, `mixed`, low contrast, `3000x80`, `80x3000`, screenshot,
dense UI), followed by `en` again. The post-large `en` checkpoint was
`605,151,232 B` private / `505,942,016 B` working set, below the preceding
large-page high-water (`605,732,864 B` / `506,232,832 B`). This verifies that
completed large-image transient storage is released/reused rather than
retained as an ever-growing per-request allocation.

The synthetic dense UI corpus still has a known CPU/GPU result-parity failure
caused by a subpixel DB-component boundary becoming a different tiny crop.
It remains an explicit regression target, not a supported parity claim and
not a condition that permits CPU fallback. Production medium detector
`1x3x160x704` likewise remains driver-unqualified on this adapter and reports
`VK_ERROR_DEVICE_LOST` fail-closed. These UMA measurements are correctness
and memory-lifetime evidence; they do not claim a universal performance win
over ONNX Runtime or CPU SIMD.

### 2026-08-22: Vulkan admission and multi-mode recheck

Hybrid keeps its required transfer-inclusive admission rule: a Vulkan segment
is used only after the same complete boundary is measured no slower than the
CPU SIMD segment.  The optional complete-GPU-graph experiment is now read once
when `OCR` is constructed; it cannot accidentally supersede the normal
per-operator no-slower policy during a request.  This protects hybrid latency
and CPU utilization on UMA adapters, where a valid complete Vulkan graph can
still be slower than the CPU path.

On the local Radeon 780M / AVX-512 host, `mixed.ppm` (three text regions,
two warm-ups/five timed runs) measured the following C++ implementation
values:

| Model | CPU-only | Hybrid | GPU-only |
| --- | ---: | ---: | ---: |
| tiny | `15.616 ms` | `15.888 ms` | `60.750 ms` |
| small | `41.700 ms` | `42.537 ms` | `173.970 ms` |
| medium | `175.402 ms` | `201.552 ms` | unavailable on this production OCR shape |

Hybrid tracing for medium also observed two eligible Vulkan segments: crop
RGB+Conv.0 (`1.426` vs `1.632 ms`, GPU/CPU) and one terminal CTC GEMM
(`5.438` vs `5.636 ms`). A slower same-model CTC shape (`5.058` vs
`4.350 ms`) remained on CPU. This is the intended behavior: Vulkan lowers
CPU work when it is equal or faster, while slower local segments do not make
the request slower.

The revised strict Vulkan regression passed tiny and small six-image OCR
corpora (two repeats, 63 records each), the kernel smoke, and a true tiny
N=2 detector/recognizer batch (`98.783 -> 91.599 ms`, `1.078x`). The
small-to-large-to-small GPU memory ladder retained a bounded final `en`
checkpoint of `605,863,936 B` private / `507,125,760 B` working set after
the screenshot and dense-UI phases. These are local results, not an
unsubstantiated claim of universal superiority over ONNX Runtime; a matched
ORT benchmark on the target GPU remains required before making that claim.

### 2026-08-22: GPU-only replay result consistency

The strict GPU-only replay path now applies the same compact CTC metadata
convention as the non-replayed GPU path: blank and repeated CTC rows have
zero emitted-character probability. This is performed only after the Vulkan
CTC reduction has downloaded its `[N,T]` result records; logits are never
downloaded and no CPU neural operator is introduced. It prevents a page with
multiple replayed recognition-width graphs from reporting a confidence that
differs merely because its command buffers were coalesced into one queue
submission.

During the same requalification, the approximate AVX2/AVX-512 vector-exp
sigmoid was changed to an explicit deployment opt-in
(`PPOCR_ENABLE_AVX512_SIGMOID` or `PPOCR_ENABLE_AVX2_SIGMOID`). The default
uses the exact scalar-libm logistic expression because detector probabilities
feed the DB threshold boundary. This restores the tight kernel numerical
check while preserving optional ISA experimentation for deployments that have
qualified its FP32 drift.

Fresh release checks on the local AMD Radeon 780M / LLPC stack:

| Check | Result |
| --- | --- |
| Backend probe | Vulkan loader/compute/full graph: `1/1/1`; device `AMD Radeon 780M Graphics` |
| Kernel smoke | passed |
| Detector graph, tiny `1x3x160x704` | max error `5.63276e-7`; CPU/GPU mean `10.798 / 32.052 ms` |
| Detector graph, small `1x3x160x704` | max error `3.05939e-7`; CPU/GPU mean `20.878 / 51.629 ms` |
| Detector graph, medium `1x3x160x640` | max error `7.56991e-8`; CPU/GPU mean `103.525 / 290.357 ms` |
| GPU-only OCR, tiny | 7 images × 2 runs; 64 records; exact text/geometry; max confidence delta `0.0011663` |
| GPU-only OCR, small | 7 images × 2 runs; 64 records; exact text/geometry; max confidence delta `0.0002261` |
| GPU-only replay stress, tiny screenshot (`rec_batch=2`) | 3 runs; 34 records; exact text/geometry; max confidence delta `0.0003594` |

The qualified image set is `16x16`, English, mixed-script, low-contrast,
ultra-wide, ultra-tall and `1920x1080` screenshot pages. The production
medium OCR detector shape remains fail-closed on this Radeon/LLPC driver with
`VK_ERROR_DEVICE_LOST`; it is a driver qualification limitation, not a CPU
fallback. These results establish complete GPU-only execution and replay
consistency, but they are not a claim that this UMA Vulkan implementation
outperforms ONNX Runtime.

### 2026-08-22: persistent replay activation-release regression

The strict Vulkan replay cache now owns the full set of arena slots captured
while its command buffer is recorded, including private intermediate
activations and manually pinned RGB/input slots.  Dropping a replay graph (on
model release or optional cache eviction) unpins and releases this complete
set before reusable arena storage is reclaimed.  This closes a device-memory
lifetime gap where only public input/output descriptors were previously
released.

`ppocr_gpu_graph_mem_soak` now accepts `--device-input-replay`.  Unlike the
ordinary host-input memory ladder, this mode forces persistent command-buffer
record/replay, releases the model after every shape, and requires both
`live_bytes` and `allocated_bytes` to be zero at each `replay-release`
checkpoint.  It is a logical Vulkan-arena test; Windows process
`PrivateUsage` can still retain allocator high-water pages and must not be
misreported as immediately returned physical memory.

Validated with the release build on AMD Radeon 780M / LLPC:

| Model | Device-input replay ladder | Result |
| --- | --- | --- |
| tiny detector | `1x3x64x128 -> 96x256 -> 128x512 -> 64x128` | every release: `live=0`, `allocated=0` |
| small detector | `1x3x64x128 -> 96x256 -> 128x512 -> 64x128` | every release: `live=0`, `allocated=0` |

The normal small-to-large-to-small host-input ladder also remains green:
tiny reaches a `46,006,272`-byte reusable arena high-water and returns after
every sample to its `1,713,924`-byte immutable baseline; small reaches
`73,007,104` bytes and returns to `9,813,844` bytes.  Capacity is deliberately
reusable, while the immutable baseline proves no dynamic activation stays
live after the largest shape.

Reproduce the replay lifetime test:

```powershell
$models = 'D:\workprj\aicoder\.tmp\ocr-models'
.\build-gpu-only-ep\ppocr_gpu_graph_mem_soak.exe `
  "$models\ppocrv6_tiny_det.onnx" --device-input-replay `
  1 3 64 128 1 3 96 256 1 3 128 512 1 3 64 128
```

The revised batch fallback keeps the GPU-only contract strict: a rejected
N-batch is remembered for that exact shape and then retried as independent
N=1 Vulkan pages.  It does not attempt a speculative runtime reset and never
falls through to CPU neural inference.  Current same-page N=2 checks remain
serial-equivalent: tiny `89.904 -> 84.510 ms` (`1.064x`) and small
`247.571 -> 236.356 ms` (`1.047x`).  Production medium at its normal OCR
shape is still driver-unqualified on this adapter and fails closed with
`VK_ERROR_DEVICE_LOST`; no successful medium GPU-batch claim is made.

### 2026-08-22: three-size diverse CPU/ONNX Runtime sample

`bench/compare_models.ps1` now defaults to a broader real-image set: tiny
input, single word, English, Chinese, mixed-script, low contrast, wide/tall
strips, and `1920x1080`/`2560x1440` screenshots.  This makes the standard
benchmark cover scale and aspect-ratio changes instead of only a short
English line.

For a quick, reproducible CPUExecutionProvider comparison, the release build
was measured with one warm-up and one sample each on `en`, `mixed`, and
`3000x80` wide-strip pages (16 native workers, ONNX Runtime 1.28.0 CPU EP,
same detector/recognizer ONNX files and pipeline semantics).  The captured
JSON is `build-gpu-only-ep/cpu_ort_sampled_20260822.json`.

| Model | Native total | ORT total | Latency improvement | Worst sampled case |
| --- | ---: | ---: | ---: | ---: |
| tiny | `45.357 ms` | `301.243 ms` | `84.94%` | `81.02%` |
| small | `116.614 ms` | `494.681 ms` | `76.43%` | `65.35%` |
| medium | `475.979 ms` | `842.178 ms` | `43.48%` | `16.34%` |

These are CPU-only results, so they demonstrate the native SIMD/fused path
against the available matched ORT CPU provider rather than incorrectly
attributing the result to Vulkan.  The short single-sample run is useful as
a cross-model sanity check, not a replacement for a longer deployment
benchmark.  GPU-only tiny/small and hybrid retain their independently tested
Vulkan policy; medium GPU-only continues to fail closed on the tested
Radeon/LLPC production shape.

### 2026-08-22: hybrid admission and GPU residency tightening

Hybrid admission now uses one strict rule for its RGB preprocessing, fused
detector-stem, recognizer-stem and graph-level Vulkan boundaries: measured
upload + dispatch + readback must be `GPU <= CPU`.  The earlier 15% latency
grace for three RGB/stem paths was removed, so hybrid cannot trade a slower
request merely to move work from CPU.  Equal-or-faster GPU segments are still
preferred, exactly as the public backend contract requires.

On the Radeon 780M / LLPC trace, the small mixed page correctly rejected the
slower GPU detector stem (`8.360 / 6.009 ms`), recognizer stem
(`1.130 / 0.892 ms`) and CTC projections.  A medium `1920x1080` trace also
admitted only qualifying CTC shapes (for example `15.470 / 21.808 ms`), while
leaving slower shapes on CPU.  Thus hybrid lowers CPU use only where the full
boundary is non-regressive; it is not a blanket Vulkan offload mode.

`PPOCR_GPU_RECLAIM_TRANSIENT=0|1` is now the common retention switch for both
GPU-only and optional complete-graph hybrid execution; the old
`PPOCR_GPU_ONLY_RECLAIM_TRANSIENT` spelling remains supported.  The default
is `1`.  A GPU-only tiny multi-size ladder (`16x16`, normal, wide, tall,
1080p, 1440p, then `16x16`) has been captured for the default and explicit
zero-retention policies in `build-gpu-only-ep/tiny_gpu_memory_diverse_20260822.json`
and `build-gpu-only-ep/tiny_gpu_memory_diverse_retain0_20260822.json`.
Windows process accounting still retains allocator high-water pages, so the
authoritative per-graph lifetime gate remains the Vulkan arena's
`live_bytes`/`allocated_bytes` replay regression rather than PrivateUsage
alone.

### 2026-08-22: runtime-generation-safe hybrid admission

The RGB/stem hybrid admission caches are now keyed by the Vulkan runtime
generation as well as image and operator shape.  A device-loss recovery tears
down arena buffers and can select a different physical-device configuration;
reusing a pre-recovery yes/no verdict would be unsound.  The next request now
re-runs its transfer-inclusive qualification automatically, while normal
same-generation requests remain a fast cache lookup.

An AVX-512 sigmoid A/B was also rechecked.  On the 16-worker English sample
it did not show a stable all-model speedup (tiny `14.769 ms`, small
`33.743 ms`, medium `145.255 ms`), so it stays opt-in.  Tiny and small
eight-image GPU-only parity runs still passed when the CPU-side option was
present (exact text/geometry; confidence deltas `0.0011663` and `0.0002261`).
Keeping exact default detector sigmoid protects DB threshold boundaries while
leaving a qualified deployment an explicit SIMD option.