Files
reader/web/test/sync.test.ts
T
CodeBuddy 78bfe557c3 feat: 客户端 Android 与 H5 双端,含精排渲染层
按产品设计 §5「原生外壳 + WebView 渲染」落地客户端,Android 与浏览器共用同一份
渲染层;同时把服务端的格式范围从 epub + pdf 扩展到 txt 与漫画。

## 为什么两端共用一份渲染层

Android 端不重写渲染。它是一个 Kotlin 原生外壳,只负责文件缓存、手势、账号与书架,
渲染复用 web/ 构建出的同一份产物。原因很直接:排版策略一旦有第二份实现就一定会分叉,
而「忠于精排」正是这个产品的差异点。

共用的抽象在 web/src/core/platform.ts(传输、键值存储、二进制存储、连通性)。
外壳只覆盖 WebView 做不到或做不好的三件事:

- 连通性:WebView 里的 navigator.onLine 只要有网卡就报「在线」,连上没有出口的
  Wi-Fi 时会骗人,客户端于是转圈而不是读缓存。改用平台的 NET_CAPABILITY_VALIDATED。
- 稳定的设备名:UA 里的型号会随 Chrome 升级变化,作为「上次在哪个设备读的」标签不可靠。
- 原生提示:一行 Kotlin 的 Toast,不用在 Web 层造通知 UI。

跨语言的接缝(Kotlin 桥接 ↔ TypeScript 声明)没法互相类型检查,因此加了
android/scripts/check-shell.sh 做双向契约校验,并接进 CI。

## 精排 CSS 策略

这是整个项目最产品化的一块 CSS,所以规则是写明的而不是凑出来的。

三条原则:不设任何属性除非不设会坏;每个可调项都是默认 inherit 的可继承自定义属性
(「关」的意思是「不动它」,不是「用我们的默认值」);任何改变作者意图的行为都是
用户主动开启且持久化。

实际 override 只有七处,每处都在注释里写了为什么:图片/表格/pre 在窄屏会撑破布局、
ruby 振假名在 16px 下按默认 50% 太小、长中文标题不换行会溢出。除此之外不动。

被刻意删掉的做法:reset 样式表、统一的「阅读主题」、给 p 加边距。它们能让排版
「看起来一致」,而一致性恰恰是这里要避免的。

书的样式装进 shadow root。书里一条 p { color: red } 不能改到书架,App 的 .topbar
也不能被书撞上。用 iframe 也能隔离,但每次换章要重建文档,会丢滚动位置并在 Android
上多花几倍内存。关键是**不往 shadow root 里注入任何 reset**。

## 格式层

服务端只负责流式给出原始文件,拆包、编码、章节与分页都是渲染层的事——
这些恰好是产品要自己拿主意的部分。

- EPUB:OPF/spine/nav 解析,作者样式全部保留,相对引用改写成内部 reader-res: 协议,
  绝对 URL 原样保留交给 CSP 挡掉。加载器按文档逐个容错,一个坏章节不会毁掉整本书。
- TXT:先严格校验 UTF-8(自己实现,不用宽松模式——宽松模式永远「成功」,于是永远
  发现不了解错了),失败再依次试 GB18030 与 Big5,并把选择告诉用户、允许覆盖。
  章节只能从标题推断,策略是宁可少认不要错认:800 字的段落以「第三章」开头不算标题,
  正文里提到「他在第二章读过」也不算。两条都有测试固定。
- CBZ / 图片目录:自然序排页(1,2,10 而不是 1,10,2),一页一个 fixed-layout section。
  排错不是观感问题,是故事被静默打乱。
- 检测以魔数优先:本地库里 .txt 其实是 EPUB、.cbz 里其实是 PDF 的情况很多,
  按扩展名开错加载器只会得到一个看不懂的报错。

服务端扫描器相应扩展:txt/cbz 直接索引;图片目录按文件夹识别为一本书,
文件夹路径作为主键锚点,所以**往漫画文件夹加一页不会丢进度**——这条有测试。

## 客户端三态与离线队列

离线队列记的是**脏集合**(哪些记录还没送到服务端),而不是「当前全部状态」。
两者只在一种情况下结果不同,而这种情况一定会发生:记录在请求发出之后、响应到达之前
被修改了。脏集合下这次更新仍是脏的,要等它自己的确认才清除;「全发」模式下它会被
那个更早的响应顺手清掉,然后永远送不到服务端——一个丢失的阅读位置,是这个产品
最不能有的失败。

阅读位置格式 r1:<偏移>:<章节 id>。三点都是踩过之后定的:版本前缀(服务端把 locator
当不透明字符串长期保存,客户端将来改格式必须能认出「这不是我认识的格式」)、
章节 id 放最后(它是文件路径,会含冒号,只从前两个冒号切才不会丢数据)、
偏移是章节内的比例而非像素或列号(列号会随字号变化,于是读者一调字号位置就跳)。

TXT、漫画、PDF 的支持也一并接上,PDF 用浏览器自带阅读器保底可读。

## 服务端

- GET /books/:id/file?path=… :按页取图。path 必须是这本书已登记的 rel_path,
  服务端按表精确匹配,不用 query 拼路径,所以穿越只会匹配到空。
- CORS 白名单(CORS_ORIGINS,默认回显任意来源)+ H5 静态托管(WEB_DIR)。
  托管在同一容器里,这才保住「一条命令跑起来」。
- Android 用 WebViewAssetLoader 从 https://appassets.androidplatform.net/ 提供
  打包好的客户端,而不是 file://。后者发出的 fetch 是不透明的跨源请求,所有访问
  服务端的调用都会被拦,而通常的绕法是把 web security 关掉——那等于把 WebView
  变成一扇敞开的门。用资产加载器拿到的是正常安全源,请求按普通 CORS 走,
  isSecureContext 为真,IndexedDB 等行为与浏览器一致。这一个选择是「一份 bundle
  两个宿主」能成立的前提。

## 过程中发现并修掉的问题

- **ETag 里塞了中文路径**:HTTP 头值必须是 ISO-8859-1,漫画页码路径是中文时会抛
  ERR_INVALID_CHAR,把一个正常的取图请求变成 500。改成对路径取散列。
- **漫画文件夹的主键用了页集合散列**:注释写的是「加一页不产生新书」,实现却是散列
  页集合,加一页就换 id、进度全丢。改成用文件夹路径作为锚点,和书用 dc:identifier
  作锚点是同一个道理。测试固定住了。
- **locator 解析与写出不对称**:写出是两段、解析却按三段切,导致位置根本存不回来。
  章节 id 含冒号时更糟。
- **离线队列的清理条件**:原本按「清空全部」处理,会把请求在途时产生的新位置一并
  抹掉。改成只清那一批、且只清时间戳吻合的条目。
- **测试自身的错误**:并发刷新去重的用例里,我一开始用 resolveRefresh?.() 调一个
  类型已收窄成 never 的变量,断言等于什么都没测;「outbox 为空时不推」的用例里把
  登录的 POST 也数了进去,断言恒不成立。两处都是测试写错了。
- **凭据用错了**:一个权限用例里我用 createAsAdmin 建账号却拿另一组凭据登录,
  401 被当成「越权被拒」而通过。这也是测试写错了。

## 验证方式

```
cd server && npm run typecheck && npm test && npm run build   # 37/37 通过
cd web    && npm run typecheck && npm test && npm run build   # 147/147 通过
sh android/scripts/check-shell.sh                             # 桥接契约双向校验通过
```

另用真实 HTTP 请求端到端验证过:注册 → 扫描 → 书架(四种格式全部可见)→ 漫画按页
取图(含中文路径 ETag)→ 列表页 → 进度同步 → 继续阅读;H5 静态托管、SPA 回落、
CORS 回显、路径穿越拒绝均按预期。

Android 壳本轮**没有真机编译验证**:环境里没有 JDK 和 Android SDK。Kotlin 侧做了
静态校验(包名与目录一致、项目内 import 可解析、桥接方法与 web 侧声明双向对齐),
并接进了 CI。第一次真机构建可能需要微调 Gradle 版本或 AGP 版本。

## 说明

- docker-compose.yml 与 Dockerfile 已改为从仓库根构建并打包 H5 产物;
  build 上下文变了,如果之前用了 `build: ./server` 需要改成根目录。
- iOS 未做设计(按设计文档 §7,等有赞助再说)。
- 移动端批注输入仍未做(先保证读,后补写)。

Ref: #1
2026-09-16 14:49:11 +08:00

386 lines
16 KiB
TypeScript

import { describe, expect, it } from 'vitest';
import { ReaderApi } from '../src/api/client.ts';
import { ApiError } from '../src/api/errors.ts';
import { SyncEngine } from '../src/core/sync.ts';
import { OfflineStore, isOutboxEmpty } from '../src/store/offline.ts';
import { FakeTransport, MemoryKv, makePlatform } from './helpers/env.ts';
const SESSION = {
user: { id: 'u1', username: 'me', displayName: '我', role: 'member' as const, createdAt: 0 },
accessToken: 'access-1',
accessTokenExpiresAt: Date.now() + 3_600_000,
refreshToken: 'refresh-1',
refreshTokenExpiresAt: Date.now() + 86_400_000,
};
function sessions() {
const kv = new MemoryKv();
return {
kv,
store: {
async load() {
const raw = await kv.get('s');
return raw ? JSON.parse(raw) : null;
},
async save(session: unknown) {
await kv.set('s', JSON.stringify(session));
},
async clear() {
await kv.remove('s');
},
},
};
}
function build() {
const transport = new FakeTransport();
const platform = makePlatform(transport);
const session = sessions();
const api = new ReaderApi(platform, session.store);
api.setBaseUrl('http://test.local');
const offline = new OfflineStore(platform.kv);
const engine = new SyncEngine(api, offline, platform);
return { transport, platform, api, offline, engine };
}
describe('ReaderApi', () => {
it('attaches the bearer token once a session exists', async () => {
const { api, transport } = build();
transport.json(SESSION);
await api.login('me', 'password12');
transport.json({ items: [], total: 0, page: 1, pageSize: 50 });
await api.listBooks();
const last = transport.requests.at(-1)!;
expect(last.headers['authorization']).toBe('Bearer access-1');
});
it('does not send a token to the public endpoints', async () => {
const { api, transport } = build();
transport.json({ name: 'reader', apiVersion: 1, registrationOpen: true, userCount: 0 });
await api.instance();
expect(transport.requests[0]!.headers['authorization']).toBeUndefined();
});
it('refreshes an expired token once and retries the original request', async () => {
const { api, transport } = build();
transport.json(SESSION);
await api.login('me', 'password12');
transport.respondWith((request) => {
if (request.url.endsWith('/api/v1/auth/refresh')) {
return { status: 200, headers: {}, json: { ...SESSION, accessToken: 'access-2', refreshToken: 'refresh-2' } };
}
if (request.headers['authorization'] === 'Bearer access-1') {
return { status: 401, headers: {}, json: { error: { code: 'TOKEN_EXPIRED', message: 'expired' } } };
}
return { status: 200, headers: {}, json: { items: [], total: 0, page: 1, pageSize: 50 } };
});
await api.listBooks();
const refreshCalls = transport.countMatching((request) => request.url.endsWith('/auth/refresh'));
expect(refreshCalls).toBe(1);
expect(transport.requests.at(-1)!.headers['authorization']).toBe('Bearer access-2');
});
it('does not refresh or retry when the token signature is invalid', async () => {
// `TOKEN_INVALID` means refreshing cannot help. Retrying would burn the
// single-use refresh chain on a request that can never succeed.
const { api, transport } = build();
transport.json(SESSION);
await api.login('me', 'password12');
transport.json({ error: { code: 'TOKEN_INVALID', message: 'bad signature' } }, 401);
await expect(api.listBooks()).rejects.toMatchObject({ code: 'TOKEN_INVALID' });
expect(transport.countMatching((request) => request.url.endsWith('/auth/refresh'))).toBe(0);
});
it('de-duplicates concurrent refreshes so a rotation is not burned twice', async () => {
// The server rotates refresh tokens (single-use). Three parallel requests
// hitting an expired token must share one refresh, or the second and third
// would present a token that was already consumed and sign the reader out.
const { api, transport } = build();
transport.json(SESSION);
await api.login('me', 'password12');
let refreshCount = 0;
let releaseRefresh = (): void => undefined;
const refreshGate = new Promise<void>((resolve) => {
releaseRefresh = resolve;
});
transport.respondWith(async (request) => {
if (request.url.endsWith('/auth/refresh')) {
refreshCount += 1;
await refreshGate;
return { status: 200, headers: {}, json: { ...SESSION, accessToken: 'access-2', refreshToken: 'refresh-2' } };
}
if (request.headers['authorization'] === 'Bearer access-1') {
return { status: 401, headers: {}, json: { error: { code: 'TOKEN_EXPIRED', message: 'expired' } } };
}
return { status: 200, headers: {}, json: { items: [], total: 0, page: 1, pageSize: 50 } };
});
const inFlight = Promise.all([api.listBooks(), api.listBooks(), api.listBooks()]);
await new Promise((resolve) => setTimeout(resolve, 0));
releaseRefresh();
await inFlight;
expect(refreshCount).toBe(1);
});
it('clears the session when the refresh token itself is rejected', async () => {
const { api, transport } = build();
transport.json(SESSION);
await api.login('me', 'password12');
transport.respondWith((request) =>
request.url.endsWith('/auth/refresh')
? { status: 401, headers: {}, json: { error: { code: 'REFRESH_INVALID', message: 'used' } } }
: { status: 401, headers: {}, json: { error: { code: 'TOKEN_EXPIRED', message: 'expired' } } },
);
await expect(api.listBooks()).rejects.toBeInstanceOf(ApiError);
expect(api.currentSession()).toBeNull();
});
it('reports an unreachable server as an offline error, not a server error', async () => {
// A rejected fetch is what a LAN address that is not reachable looks like,
// and the whole three-state design keys off this being `offline` so the UI
// can silently fall back to cached content instead of showing an error.
const transport = new FakeTransport();
transport.failWith(new TypeError('Failed to fetch'));
const platform = makePlatform(transport);
const session = sessions();
const api = new ReaderApi(platform, session.store);
api.setBaseUrl('http://test.local');
await expect(api.instance()).rejects.toMatchObject({ kind: 'offline' });
});
it('treats a non-JSON 200 as a connectivity problem rather than crashing', async () => {
// A captive portal or a misconfigured reverse proxy answers 200 with HTML.
// Parsing that as JSON would throw a SyntaxError and lose the offline path.
const transport = new FakeTransport();
transport.nonJson();
const platform = makePlatform(transport);
const api = new ReaderApi(platform, sessions().store);
api.setBaseUrl('http://test.local');
await expect(api.instance()).rejects.toBeInstanceOf(ApiError);
});
});
describe('OfflineStore', () => {
it('keeps the newer progress and discards a stale write', async () => {
const store = new OfflineStore(new MemoryKv());
await store.setProgress({ bookId: 'b1', locator: 'r1:0.5:c1', percentage: 0.5, chapterTitle: 'c1', device: 'a', updatedAt: 2000 });
await store.setProgress({ bookId: 'b1', locator: 'r1:0.1:c1', percentage: 0.1, chapterTitle: 'c1', device: 'a', updatedAt: 1000 });
expect(store.current.progress['b1']!.percentage).toBe(0.5);
});
it('does not let a stale pull overwrite newer local progress', async () => {
const store = new OfflineStore(new MemoryKv());
await store.setProgress({ bookId: 'b1', locator: 'r1:0.9:c1', percentage: 0.9, chapterTitle: 'c1', device: 'a', updatedAt: 5000 });
await store.applyPulled({
serverTime: 100,
progress: [{ bookId: 'b1', locator: 'r1:0.2:c1', percentage: 0.2, chapterTitle: 'c1', device: 'b', updatedAt: 1000 }],
notes: [],
});
expect(store.current.progress['b1']!.percentage).toBe(0.9);
});
it('records a note deletion as a pending tombstone, not a lost write', async () => {
const store = new OfflineStore(new MemoryKv());
await store.upsertNotes([
{ id: 'n1', bookId: 'b1', type: 'highlight', locator: 'x', text: 't', comment: '', color: '', updatedAt: 1 },
]);
await store.deleteNote('n1');
expect(store.notesFor('b1')).toHaveLength(0);
expect(store.current.pendingNoteDeletes).toContain('n1');
});
it('clears a pending tombstone when the same note is written again', async () => {
const store = new OfflineStore(new MemoryKv());
await store.deleteNote('n1');
await store.upsertNotes([
{ id: 'n1', bookId: 'b1', type: 'note', locator: 'x', text: 't', comment: '', color: '', updatedAt: 50 },
]);
expect(store.current.pendingNoteDeletes).not.toContain('n1');
});
it('recovers from a corrupt snapshot instead of failing to start', async () => {
const kv = new MemoryKv();
await kv.set('reader.offline.v1', '{not json');
const store = new OfflineStore(kv);
const snapshot = await store.load();
expect(snapshot.books).toEqual({});
expect(snapshot.serverTime).toBe(0);
});
it('serialises concurrent writes so a fast reader does not lose a position', async () => {
const store = new OfflineStore(new MemoryKv());
await Promise.all(
Array.from({ length: 20 }, (_value, index) =>
store.setProgress({
bookId: `b${index}`,
locator: `r1:0:s${index}`,
percentage: index / 20,
chapterTitle: 'x',
device: 'a',
updatedAt: 1000 + index,
}),
),
);
await store.flush();
expect(Object.keys(store.current.progress)).toHaveLength(20);
});
});
describe('SyncEngine', () => {
it('pushes the outbox and then pulls only the delta', async () => {
const { api, transport, offline, engine, platform } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
await offline.setProgress({ bookId: 'b1', locator: 'r1:0.5:c1', percentage: 0.5, chapterTitle: 'c1', device: platform.deviceLabel, updatedAt: 1000 });
const seen: string[] = [];
transport.respondWith((request) => {
seen.push(`${request.method} ${request.url}`);
if (request.method === 'POST' && request.url.endsWith('/api/v1/sync')) {
return { status: 200, headers: {}, json: { accepted: 1, rejected: 0, serverTime: 5000, progress: [], notes: [] } };
}
return { status: 200, headers: {}, json: { serverTime: 5000, progress: [], notes: [] } };
});
await engine.syncNow();
expect(seen[0]).toBe('POST /api/v1/sync');
// The pull reuses the cursor the push just returned. That is intentional: a
// pull from 0 would re-download the entire server state after every page
// turn, which is exactly what the incremental cursor exists to avoid.
expect(seen[1]).toContain('since=5000');
expect(isOutboxEmpty(offline.current)).toBe(true);
expect(offline.current.serverTime).toBe(5000);
});
it('sends nothing but a pull when the outbox is empty', async () => {
const { api, transport, offline, engine } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
transport.json({ serverTime: 10, progress: [], notes: [] });
await engine.syncNow();
// Counting only /sync: the login above is itself a POST, and counting all
// POSTs would make this assertion vacuous.
const syncPosts = transport.countMatching(
(request) => request.method === 'POST' && request.url.endsWith('/api/v1/sync'),
);
expect(syncPosts).toBe(0);
});
it('uses the stored cursor for the next pull', async () => {
const { api, transport, offline, engine } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
transport.json({ serverTime: 777, progress: [], notes: [] });
await engine.syncNow();
transport.json({ serverTime: 999, progress: [], notes: [] });
await engine.syncNow();
const pulls = transport.requests.filter((request) => request.method === 'GET' && request.url.includes('/sync?'));
expect(pulls[0]!.url).toContain('since=0');
expect(pulls[1]!.url).toContain('since=777');
});
it('degrades to offline without surfacing an error', async () => {
const { api, transport, offline, engine, platform } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
platform.setOnline(false);
await engine.syncNow();
expect(engine.status().state).toBe('offline');
expect(engine.status().message).toContain('离线');
});
it('does not lose the outbox when the network fails mid-push', async () => {
const { api, transport, offline, engine, platform } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
await offline.setProgress({ bookId: 'b1', locator: 'r1:0.5:c1', percentage: 0.5, chapterTitle: 'c1', device: platform.deviceLabel, updatedAt: 1000 });
transport.failWith(new Error('boom'));
await engine.syncNow();
// The position must still be there to send on the next attempt.
expect(isOutboxEmpty(offline.current)).toBe(false);
expect(offline.current.progress['b1']).toBeDefined();
});
it('chunks a large outbox instead of sending an oversized batch', async () => {
const { api, transport, offline, engine, platform } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
for (let index = 0; index < 2500; index += 1) {
await offline.setProgress({
bookId: `b${index}`,
locator: `r1:0:s${index}`,
percentage: 0,
chapterTitle: '',
device: platform.deviceLabel,
updatedAt: 1000,
});
}
const sizes: number[] = [];
transport.respondWith((request) => {
if (request.method === 'POST') {
const body = JSON.parse(request.body ?? '{}') as { progress?: unknown[] };
sizes.push(body.progress?.length ?? 0);
return { status: 200, headers: {}, json: { accepted: 1, rejected: 0, serverTime: 1, progress: [], notes: [] } };
}
return { status: 200, headers: {}, json: { serverTime: 1, progress: [], notes: [] } };
});
await engine.syncNow();
expect(sizes.length).toBeGreaterThan(1);
expect(Math.max(...sizes)).toBeLessThanOrEqual(1000);
expect(sizes.reduce((sum, size) => sum + size, 0)).toBe(2500);
});
it('reports rejection counts without blocking later flushes', async () => {
const { api, transport, offline, engine, platform } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
await offline.setProgress({ bookId: 'gone', locator: 'x', percentage: 0, chapterTitle: '', device: platform.deviceLabel, updatedAt: 1 });
transport.respondWith((request) =>
request.method === 'POST'
? { status: 200, headers: {}, json: { accepted: 0, rejected: 1, serverTime: 2, progress: [], notes: [] } }
: { status: 200, headers: {}, json: { serverTime: 2, progress: [], notes: [] } },
);
await engine.syncNow();
// A record the server will never accept must not stay in the outbox
// forever, or every later position would queue behind it.
expect(isOutboxEmpty(offline.current)).toBe(true);
expect(engine.status().message).toContain('拒绝');
});
it('does nothing when there is no session', async () => {
const { engine } = build();
await engine.syncNow();
expect(engine.status().state).toBe('signed-out');
});
it('does not run two syncs at once', async () => {
const { api, transport, offline, engine } = build();
transport.json(SESSION);
await api.login('me', 'password12');
await offline.load();
let concurrent = 0;
let peak = 0;
transport.respondWith(async () => {
concurrent += 1;
peak = Math.max(peak, concurrent);
await new Promise((resolve) => setTimeout(resolve, 5));
concurrent -= 1;
return { status: 200, headers: {}, json: { serverTime: 1, progress: [], notes: [] } };
});
await Promise.all([engine.syncNow(), engine.syncNow(), engine.syncNow()]);
expect(peak).toBe(1);
});
});