mirror of
https://github.com/RapidAI/RapidOCR.git
synced 2026-11-11 13:30:51 +08:00
Cursor AgentandDaniel
3622c4598e
Expose the PP-OCRv6 engine through a stable C ABI and installable shared and static libraries, plus a ctypes Python package that follows the RapidOCR call shape. Keep AVX-512 out of the AVX2 object file and speed up GEMM accumulate and the AArch64 GEMM tile. Co-authored-by: Daniel <znsoft@163.com>
102 lines
5.0 KiB
Markdown
102 lines
5.0 KiB
Markdown
# CPU performance
|
||
|
||
The hot path is the built-in ONNX interpreter: convolutions, GEMM, and the
|
||
RGB-to-NCHW front end. Post-process (DB boxes, CTC decode) is small next to
|
||
those kernels. Threads come from a persistent pool (`PPOCR_BENCH_THREADS`,
|
||
default `min(16, hardware_concurrency)`) plus the recognizer's outer batch
|
||
pool, which is capped by the hardware thread count.
|
||
|
||
## One binary, three targets
|
||
|
||
| Target | Where the code lives | When it runs |
|
||
| --- | --- | --- |
|
||
| x86 without AVX (scalar / SSE2) | `src/kernels.cpp`, compiled with `-mno-avx -mno-avx2 -mno-avx512f` | no AVX2 |
|
||
| x86 AVX2 + FMA, no AVX-512 | `src/kernels_avx2.cpp`, compiled with `-mavx2 -mfma -mno-avx512f -mno-avx512bw -mno-avx512vl -mno-avx512dq` | `avx2` and `fma` CPUID bits, and `PPOCR_FORCE_ISA` is not `scalar` |
|
||
| x86 AVX-512 | `src/kernels_avx512.cpp`, compiled with `-mavx512f -mavx512dq -mavx512bw -mavx512vl -mbmi -mbmi2 -mfma` | AVX-512F/DQ/BW/VL plus AVX2, FMA, BMI1, and BMI2, and OS XSAVE state, unless `PPOCR_DISABLE_AVX512` or `PPOCR_FORCE_ISA=avx2`/`scalar` |
|
||
| AArch64 NEON | `src/kernels.cpp` (NEON is baseline on AArch64) | not forced off with `PPOCR_FORCE_ISA=scalar` |
|
||
|
||
An AVX2-only machine never executes the AVX-512 object file. `HasAvx512()`
|
||
returns false unless CPUID and XCR0 say the OS has enabled AVX-512, so
|
||
`PPOCR_FORCE_ISA=avx512` on that machine stays on AVX2 or scalar. The
|
||
`isa_encoding` test disassembles the baseline and AVX2 objects and fails if
|
||
they contain `zmm` or AVX-512 mask registers.
|
||
|
||
`PPOCR_FORCE_ISA=scalar|avx2|avx512|neon` selects a legal path for benchmarks.
|
||
It cannot turn on an ISA the CPU does not have.
|
||
|
||
## What changed
|
||
|
||
Recognizer attention uses `GemmAccumulate`. The AVX2 and AVX-512 kernels
|
||
previously streamed the weight matrix once per row. They now pair two rows
|
||
so each weight vector is loaded once and applied to two accumulators. Each
|
||
output element still walks K in ascending order. Set
|
||
`PPOCR_DISABLE_AVX2_GEMM_ACC2` or `PPOCR_DISABLE_AVX512_GEMM_ACC2` to restore
|
||
the one-row kernel.
|
||
|
||
On AArch64 the GEMM used to reload and store C on every K step (four-wide
|
||
NEON). It now keeps a 2×16 register tile, which is the same idea as the x86
|
||
kernels. `PPOCR_DISABLE_NEON_GEMM_BLOCK` restores the scalar loop.
|
||
|
||
`tools/bench_cpu.sh` and `ppocr_isa_bench` print the active ISA and time a
|
||
32×256×256 GEMM, the matching accumulate, a 960×544 identity RGB normalize,
|
||
and optionally a full OCR.
|
||
|
||
## What was measured
|
||
|
||
Host: 4 vCPU Intel Xeon (family 6, model 207) with AVX2, AVX-512F/DQ/BW/VL,
|
||
BMI1, and BMI2. `g++` 13.3. There is no ARM board and no AVX2-only CPU in
|
||
this environment. AVX2 is measured by `PPOCR_FORCE_ISA=avx2`, which keeps
|
||
`HasAvx512()` false. The AVX2 object file was disassembled and contains no
|
||
`zmm` or AVX-512 mask registers (`tests/check_isa_encoding.sh`).
|
||
|
||
### GEMM accumulate, 32×256×256, mean of 8 runs (`ppocr_isa_bench`)
|
||
|
||
| Path | gemm_accumulate |
|
||
| --- | --- |
|
||
| AVX-512, two-row (new) | 0.070 ms |
|
||
| AVX-512, one-row (`PPOCR_DISABLE_AVX512_GEMM_ACC2=1`) | 0.094 ms |
|
||
| AVX2, two-row (new) | 0.082 ms |
|
||
| AVX2, one-row (`PPOCR_DISABLE_AVX2_GEMM_ACC2=1`) | 0.125 ms |
|
||
| scalar (`PPOCR_FORCE_ISA=scalar`) | 0.186 ms |
|
||
|
||
The same-process `Gemm` timing (not the accumulate kernel) was about 0.044 ms
|
||
on AVX-512, 0.059 ms on AVX2, and 0.089 ms on scalar. Identity RGB normalize
|
||
of a 960×544 page was about 0.6–0.7 ms on every path; it is not the bottleneck.
|
||
|
||
### End-to-end PP-OCRv6 tiny, 960×240 “Hello RapidOCR 123”
|
||
|
||
`ppocr_bench`, `PPOCR_BACKEND=cpu`, 1 warmup + 3 runs, one recognized line:
|
||
|
||
| ISA | mean |
|
||
| --- | --- |
|
||
| AVX-512 | 17.3 ms |
|
||
| AVX2 (`PPOCR_FORCE_ISA=avx2`) | 136 ms |
|
||
| scalar | 233 ms |
|
||
|
||
Turning the two-row accumulate off changed this page by about 1 ms, inside
|
||
the run-to-run spread. The page is convolution-bound. The accumulate win
|
||
shows up on the GEMM shape above, which is the attention/GEMM kernel, not on
|
||
this one short line. The large AVX-512 versus AVX2 gap is the existing
|
||
AVX-512 convolution kernels, selected at runtime; it is not a claim that the
|
||
two-row accumulate produced an 8× end-to-end speedup.
|
||
|
||
### AArch64 NEON
|
||
|
||
No ARM CPU was available. `src/kernels.cpp` cross-compiles with
|
||
`aarch64-linux-gnu-g++ -O3` and the GEMM object contains `fmla`. The same
|
||
8×6906×192 recognizer-vocabulary shape was timed under `qemu-aarch64-static`:
|
||
|
||
- Standalone copy of the old NEON reload loop versus the new register tile:
|
||
166 ms → 76 ms. That is the algorithm before/after. The tile result matched
|
||
the reload loop (max abs error 0).
|
||
- In-tree `Gemm` linked against that object: register tile 56–64 ms. Checksum
|
||
and max abs matched the scalar fallback
|
||
(`PPOCR_DISABLE_NEON_GEMM_BLOCK=1`, 31–34 ms) exactly (`sum=3.171875`,
|
||
`max_abs=1.611328`). qemu-user's translator often runs scalar ARM faster
|
||
than NEON, so the scalar number is not a wall-clock result for a Cortex or
|
||
Neoverse core.
|
||
|
||
qemu-user is not cycle-accurate. It does execute the NEON instructions, and
|
||
the NEON-versus-NEON ratio matches the drop in C-matrix traffic. Full OCR was
|
||
not run under qemu.
|