mirror of
https://github.com/hectorqin/reader.git
synced 2026-11-11 05:20:54 +08:00
fix(web,server): 让服务端只交章节,并去掉成对的 sync 请求
服务端仍在做排版:`chapter-html:<n>` 自己断段、自己提升标题、自己吐 `<p>`。于是 reader 设置里的缩进和段间距只能作用于服务端已经渲染好的那一份, 流式那条路(`chapter:<n>`)拿不到段落,同一本书因为走哪条传输而看起来不一样。 客户端的 `retypePlainText` 于是把服务端断好的段落再拆一遍——同一个判断被两个 地方执行,两次的结果就会分歧。 `POST /api/v1/sync` 的响应本来就是合并后的状态(服务端在两个接口上返回同一份 `pull`),但客户端把它丢掉,然后又为同一份数据发一个 `GET`。读者每翻一页都发 一个 POST 加一个 GET,网络面板里成对出现。另外 `setState` 每次调用都广播,而 阅读页的状态监听会重绘整棵 chrome 树,一个同步循环通知三次。 现在服务端只把整章字符交出来(`text/plain`,`chapter-html:` 不截断),断段、 去段前空格、提升标题全部由 `web/src/formats/segments.ts` 一处决定;push 的返回 值直接喂给 `applyPulled`,空闲轮询才真的发 GET;`setState` 值未变就不广播。 「这一章是不是纯文本」改由 manifest 声明(item 的 `format: 'html'` → `Section.plainText` → `isPlainText()`),不再靠 `.txt-body` 嗅探:服务端现在发 裸字符,嗅探对一章节全是纯文本的章节会答「不是」,读者会看到一本没有段落、 没有缩进、样式表也没挂上的小说。`.txt-body` 标记降级为旧服务端/旧缓存的兜底。 放弃的方案:保留服务端渲染、只在客户端覆盖样式。缩进和段间距是阅读偏好,服务端 渲染就等于把每次调整变成一次往返,而这个往返正是问题本身。也考虑过让 `chapter-html:` 直接返回服务端渲染的裸 `<p>`(不带 wrapper),但那仍然有两个 地方在判断「一段是什么」。 Ref: #34
This commit is contained in:
23 files changed
+847
-429
No files matched your search
@@ -151,6 +151,22 @@ id = sha256( (dc:identifier ?? '').toLowerCase() + '\0' + sha256(fileBytes) )
|
||||
`GET /api/v1/sync?since=<上次的 serverTime>`。客户端记住上次返回的 `serverTime`,
|
||||
每次只拿增量。移动网络下重连很便宜。
|
||||
|
||||
### 一次写入只发一个请求
|
||||
|
||||
`POST /api/v1/sync` 的响应**就是合并后的状态**(服务端在两个接口上返回同一份
|
||||
`pull`)。所以推完之后**不要再拉一次**——`SyncEngine` 直接拿 push 的返回值喂给
|
||||
`applyPulled`,没有东西可推(空闲轮询)时才真的发 `GET`。
|
||||
|
||||
之前不是这样:push 的返回值被丢掉,然后为了同一份数据再发一个 `GET`。于是读者每
|
||||
翻一页都发一个 POST **加** 一个 GET,网络面板里就是成对出现的 `sync` ——这正是
|
||||
「sync 接口在不停重复」的来源。
|
||||
|
||||
### 状态只在变化时广播
|
||||
|
||||
`setState` 先比对新旧 state/message,相同就直接返回。阅读页的状态监听会重绘整棵
|
||||
chrome 树,而一个同步循环本来会通知三次(进门 `syncing`、出门 `idle`、再加一次
|
||||
多余的 `emit()`),30 秒轮询让这件事在读者什么都不做时也持续发生。
|
||||
|
||||
---
|
||||
|
||||
## 6. 认证与安全
|
||||
|
||||
@@ -35,5 +35,6 @@
|
||||
| 触控目标不小于 28px | ✅ | 9 个控件全部合格 |
|
||||
| 每个控件都有可读的名字 | ✅ | 9 个控件全部有名字 |
|
||||
| 没有横向溢出 | ✅ | document 溢出 0px |
|
||||
| 翻页不再每次都发两次 sync | ✅ | 8 次翻页 + 6 秒静默:1 次 POST / 0 次 GET(共 1 次 sync 请求) |
|
||||
|
||||
全部通过。
|
||||
@@ -130,6 +130,11 @@
|
||||
"name": "没有横向溢出",
|
||||
"ok": true,
|
||||
"detail": "document 溢出 0px"
|
||||
},
|
||||
{
|
||||
"name": "翻页不再每次都发两次 sync",
|
||||
"ok": true,
|
||||
"detail": "8 次翻页 + 6 秒静默:1 次 POST / 0 次 GET(共 1 次 sync 请求)"
|
||||
}
|
||||
],
|
||||
"failures": []
|
||||
|
||||
+28
-18
@@ -302,34 +302,44 @@ TXT 是唯一一种**没有作者排版**的书,所以它也是唯一一处「
|
||||
默认值」是对的、而不是侵入的地方。EPUB 的默认必须交给作者自己的样式表,
|
||||
覆盖它是 bug;TXT 没有作者。
|
||||
|
||||
三层分工,改任何一层之前先看清是哪一层。**排版是客户端的,服务端只负责把
|
||||
章节完整地交出来**——这条边界是这一层最重要的一条规定:
|
||||
两层分工,改任何一层之前先看清是哪一层。**排版全部在客户端,
|
||||
服务端只负责把章节完整地交出来**——这条边界是这一层最重要的一条规定:
|
||||
|
||||
- **服务端**(`server/src/indexer/formats/text-html.ts`):只做两件事。把整章
|
||||
交出来(`chapter-html:<n>` 不按 `chapter:` 那样截断),以及**在原文上**断段
|
||||
——空行断段,「上一行以句末标点结尾、且下一行以句子开头」也断段(抓取来的
|
||||
网文常常每行一段且没有空行),章节标题行提升成 `<h3>`(否则正文会变成
|
||||
`第二章 落雨雨来了。`)。缩进、段间距、去掉段前空格**都不在这里**:它们是
|
||||
每台设备各自的阅读偏好,服务端烧进响应里就等于把每次调整变成一次往返。
|
||||
- **服务端**(`server/src/indexer/formats/text.ts`):只做一件事——把**整章
|
||||
交出来**。`chapter:` 是为流式读取开的窗口,按 `size` 截断是它的功能;
|
||||
`chapter-html:<n>` 是给读者整章读的,**不截断**,内容类型是
|
||||
`text/plain`。它**不再断段、不再提升标题、不再输出 `<p>`**:断段是阅读
|
||||
决定,服务端做就等于让 windowed 那一份有段落、流式那一份没有,也让读者的
|
||||
缩进设置变成一次往返。
|
||||
- **客户端排版**(`web/src/formats/segments.ts`):`splitTextParagraphs` 是
|
||||
「一段是什么」的**唯一定义**,本地加载的 TXT、windowed 的
|
||||
`chapter:<n>`、流式的 `chunk:` 三条路都走它,所以同一本书不会因为走哪条
|
||||
`chapter-html:<n>`、流式的 `chunk:` 三条路都走它,所以同一本书不会因为走哪条
|
||||
传输看起来不一样。它同时负责**去掉段前空格**(` 正文` 里那个全角空格是
|
||||
「段前空格」的来源,留着就是和读者的缩进**叠加**)。`ReaderView` 在
|
||||
`retypePlainText` 里把服务端已经断好的章节也再过一遍这段逻辑,拼回空行再
|
||||
拆——所以服务端断得对不对不影响观感,它只是让客户端的拆分更准。
|
||||
「段前空格」的来源,留着就是和读者的缩进**叠加**),以及**章节标题行提升成
|
||||
`<h3>`**(`splitChapterHeading`,否则正文会变成 `第二章 落雨雨来了。`)。
|
||||
`ReaderView` 在 `retypePlainText` 里对拿到的章节过这一段逻辑;如果内容带着
|
||||
历史的 `.txt-body` 标记(旧服务端、旧缓存),就按块读回文本再拆,所以旧
|
||||
响应不会被当成新响应渲染。
|
||||
- **客户端样式表**(`web/src/ui/shadow.ts` 的 `TXT_STYLESHEET`):只在
|
||||
`format === 'txt'`、或章节带着 `.txt-body` 标记时注入,作用域是
|
||||
`.txt-body`。默认两字符首行缩进(`2em`,不是 `2rem`——缩进要跟字号缩放)。
|
||||
样式表里**没有任何一条规则读服务端写的属性**:「不缩进」是同一个自定义属性
|
||||
取 0,而不是第二条要和第一条保持一致的分支。
|
||||
`ReaderView.isPlainText()` 为真时注入,作用域是 `.txt-body`。默认两字符首行
|
||||
缩进(`2em`,不是 `2rem`——缩进要跟字号缩放)。样式表里**没有任何一条规则读
|
||||
服务端写的属性**:「不缩进」是同一个自定义属性取 0,而不是第二条要和第一条
|
||||
保持一致的分支。
|
||||
- **「这一章是不是纯文本」怎么判定**:靠**声明**,不靠嗅探。`manifest` 的
|
||||
content item 写 `format: 'html'` → `createStagedDoc` 把它落到
|
||||
`Section.plainText` → `isPlainText()` 认它。这条链必须完整:服务端现在发的是
|
||||
**裸字符**,没有 `.txt-body` 包裹,所以「正文里有没有 `txt-body`」这个老办法
|
||||
对一章节全是纯文本的章节会答「不是」——读者于是看到一本没有段落、没有缩进、
|
||||
样式表也没挂上的小说,而这看起来像排版偏好而不像 bug。`.txt-body` 标记只留作
|
||||
最后兜底(旧服务端/旧缓存)。
|
||||
- **阅读设置**:段落缩进、段间距、编码三行,只在 TXT 上出现;设置面板顶部的
|
||||
「快捷」一行(字号/缩进/段间距/主题)也是同一批值的另一个入口,一次点击
|
||||
走一档。前两个是写在 stage 上的自定义属性(`--reader-txt-indent` /
|
||||
`--reader-txt-para-gap`),所以调整不需要重新注入章节、也不需要请求。
|
||||
|
||||
引用保持 `chapter:<n>` 不变,新增 `format: 'html'` 字段告诉客户端取哪一份。
|
||||
`href` 同时是**阅读位置的标识**,改它会把每个现有 TXT 读者打回第一章。
|
||||
引用保持 `chapter:<n>` 不变,`format: 'html'` 字段告诉客户端取哪一份
|
||||
(`renditionRef` 据此把它换成不截断的 `chapter-html:<n>`)。`href` 同时是
|
||||
**阅读位置的标识**,改它会把每个现有 TXT 读者打回第一章。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,95 +1,28 @@
|
||||
/**
|
||||
* The `chapter-html:<n>` rendition: one chapter, whole, as a document.
|
||||
* The `chapter-html:<n>` rendition: one chapter, whole, and nothing else.
|
||||
*
|
||||
* This is a *transport* for a chapter's characters, not a typesetting pass. The
|
||||
* server's only two jobs are to hand over the entire chapter (nothing truncated)
|
||||
* and to say where its paragraphs are (a blank line, a line ending, a
|
||||
* sentence-final newline) — the boundaries are a property of the file, so they
|
||||
* are inferred once, here, against the undecoded text.
|
||||
* This file used to be a typesetting pass — it inferred paragraph boundaries,
|
||||
* promoted a heading and emitted `<p>` elements. It is not one any more, and the
|
||||
* change is deliberate: the reader's paragraph boundaries, its indent, its
|
||||
* paragraph spacing and the removal of a scraper's leading spaces are all
|
||||
* *reading* decisions. A server that baked them into a response made every one of
|
||||
* them a round trip, and it meant the same file could be typeset two different
|
||||
* ways depending on which transport happened to deliver it (the windowed
|
||||
* `chapter-html:` path got markup; the streamed `chapter:` path got characters and
|
||||
* no paragraphs at all).
|
||||
*
|
||||
* The indentation, the paragraph spacing and the removal of the leading spaces a
|
||||
* scraper wrote are *not* the server's business, and that is a deliberate change.
|
||||
* They are reading preferences: a phone and a tablet want different indents, and
|
||||
* a reader changes them while looking at the page. A server that baked them into
|
||||
* a response made every one of those changes a round trip, and it made a book
|
||||
* read windowed (`chapter:<n>`, no markup to carry the decision) look different
|
||||
* from the same book read whole. The client typesets the text it is given (see
|
||||
* `web/src/formats/segments.ts`), so the two transports agree by construction and
|
||||
* the setting costs no request.
|
||||
* So the server's job has shrunk to the one thing only it can do: hand over the
|
||||
* characters of a chapter, complete. The client typesets them — see
|
||||
* `web/src/formats/segments.ts`, which is the single definition of what a paragraph
|
||||
* is for a TXT and is shared by every path a text can reach the reader through.
|
||||
*
|
||||
* What is left here is the shape the client's shadow root expects: a *fragment*
|
||||
* wrapped the same way an EPUB chapter is (`<div class="txt-body">…`). Nothing is
|
||||
* sanitised away by `sanitiseInjectedContent`: no scripts, no absolute URLs, and
|
||||
* every character escaped.
|
||||
* The reference is kept (`chapter-html:` rather than deleting it and rewriting the
|
||||
* client's `renditionRef`), because it is opaque and format-specific by design: a
|
||||
* client that still asks for `chapter-html:` must keep working, and it does — it
|
||||
* now receives the same bytes as `chapter:`. Removing it would have been a
|
||||
* breaking change to a published reference for no gain.
|
||||
*/
|
||||
|
||||
/** Paragraphs are separated by a blank line; a single newline is a soft wrap. */
|
||||
const PARAGRAPH_BREAK = /\n{2,}/;
|
||||
|
||||
/**
|
||||
* A lone carriage return is a line break too.
|
||||
*
|
||||
* Old conversions and files off a serial console use `CR` on its own, and a
|
||||
* newline-only split leaves the whole file as one paragraph — the exact failure
|
||||
* this file exists to prevent, arriving through the one encoding path nobody
|
||||
* tests. Normalised first so every rule below sees `\n`.
|
||||
*/
|
||||
const LINE_ENDINGS = /\r\n?/g;
|
||||
|
||||
/**
|
||||
* A paragraph that ends a sentence is a paragraph even without a blank line.
|
||||
*
|
||||
* This is the one heuristic worth having for Chinese web novels: tools that scrape
|
||||
* them routinely emit every paragraph on its own line with no blank line between,
|
||||
* and treating each line as its own paragraph is the only way such a file gets
|
||||
* any indentation at all. The lookahead requires the *next* line to start a new
|
||||
* sentence (not a continuation), so a paragraph wrapped across two lines is still
|
||||
* joined rather than split down the middle.
|
||||
*/
|
||||
const SENTENCE_BREAK = /(?<=[。!?…”』】])\n(?=\s*[^\s,。!?、;:”』】)])/;
|
||||
|
||||
/**
|
||||
* Characters that begin a *continuation* rather than a sentence.
|
||||
*
|
||||
* A line starting with one of these is finishing the sentence above it: a closing
|
||||
* quote, bracket or dash carried over, or a clause opening with punctuation. The
|
||||
* lookahead in `SENTENCE_BREAK` already excludes the ones that are unambiguous;
|
||||
* this list is the same exclusions plus the closing brackets that only *look* like
|
||||
* they could start a sentence.
|
||||
*/
|
||||
const CONTINUATION_START = /^[\s,。!?、;:”』】)\u3001-\u303f\uff01-\uff5e]/;
|
||||
|
||||
/**
|
||||
* A leading heading line, promoted out of the body.
|
||||
*
|
||||
* A chapter's own slice *includes* its heading line — `splitChapters` records the
|
||||
* heading as the chapter's first line, which is what makes a locator land on the
|
||||
* chapter's title rather than one line into it. So the body handed to this file
|
||||
* starts with the title, and rendering it as an ordinary paragraph produces the
|
||||
* tell-tale `第二章 落雨雨来了。` — title and first sentence glued together with no
|
||||
* separation.
|
||||
*
|
||||
* Promoted to a real heading rather than merely split off, because it is the one
|
||||
* piece of structure a TXT actually has, and a reader scanning for a chapter
|
||||
* boundary is looking for exactly this.
|
||||
*/
|
||||
const HEADING_PATTERNS: readonly RegExp[] = [
|
||||
/^\s*第\s*[0-90-9零一二三四五六七八九十百千万两]+\s*[章回节卷部篇集]\s*[::.、\-—]?\s*.{0,60}$/,
|
||||
/^\s*(?:序章|序言|楔子|引子|前言|后记|尾声|终章|番外|附录)\s*.{0,40}$/,
|
||||
/^\s*[卷部]\s*[0-90-9零一二三四五六七八九十百千万两]+\s*.{0,60}$/,
|
||||
/^\s*Chapter\s*\d+.*$/i,
|
||||
/^\s*#{1,3}\s+.+$/,
|
||||
];
|
||||
|
||||
/** The longest line still plausibly a heading rather than prose. */
|
||||
const MAX_HEADING_LENGTH = 60;
|
||||
|
||||
function isHeading(line: string): boolean {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.length === 0 || trimmed.length > MAX_HEADING_LENGTH) return false;
|
||||
return HEADING_PATTERNS.some((pattern) => pattern.test(trimmed));
|
||||
}
|
||||
|
||||
/** Escapes the five characters that would otherwise let a file rewrite the page. */
|
||||
export function escapeHtml(text: string): string {
|
||||
return text
|
||||
@@ -99,76 +32,3 @@ export function escapeHtml(text: string): string {
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, ''');
|
||||
}
|
||||
|
||||
/**
|
||||
* Splits a chapter body into paragraph-sized strings.
|
||||
*
|
||||
* Exported because the split is the part worth testing: the escaping and the
|
||||
* wrapping below are mechanical, and every interesting failure of this file is a
|
||||
* paragraph boundary in the wrong place.
|
||||
*/
|
||||
export function splitParagraphs(body: string): string[] {
|
||||
return body
|
||||
.replace(LINE_ENDINGS, '\n')
|
||||
.split(PARAGRAPH_BREAK)
|
||||
.flatMap((block) => splitSentenceBreaks(block))
|
||||
.map((block) => block.replace(/\n/g, '').trim())
|
||||
.filter((block) => block.length > 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* Splits a block on sentence-final newlines, leaving continuations joined.
|
||||
*
|
||||
* `String.prototype.split` cannot express this on its own: the lookahead decides
|
||||
* from the *next* line whether the newline was a paragraph boundary, and a regex
|
||||
* split throws that line away with the separator. So the lines are walked, and a
|
||||
* newline is promoted to a boundary only when the line above ends a sentence and
|
||||
* the line below starts one.
|
||||
*/
|
||||
function splitSentenceBreaks(block: string): string[] {
|
||||
const lines = block.split('\n');
|
||||
const out: string[] = [];
|
||||
let current = lines[0] ?? '';
|
||||
for (let index = 1; index < lines.length; index += 1) {
|
||||
const line = lines[index] ?? '';
|
||||
const endsSentence = SENTENCE_END.test(current.trimEnd());
|
||||
const startsSentence = line.trim().length > 0 && !CONTINUATION_START.test(line);
|
||||
if (endsSentence && startsSentence) {
|
||||
out.push(current);
|
||||
current = line;
|
||||
} else {
|
||||
current += `\n${line}`;
|
||||
}
|
||||
}
|
||||
out.push(current);
|
||||
return out;
|
||||
}
|
||||
|
||||
/** True when a line ends in a mark that closes a sentence. */
|
||||
const SENTENCE_END = /[。!?…”』】]$/;
|
||||
|
||||
/**
|
||||
* Renders a chapter body as the `chapter-html` payload.
|
||||
*
|
||||
* A `<div class="txt-body">` rather than bare `<p>`s, so the client can scope its
|
||||
* own TXT rules to exactly this content and leave an EPUB's chapters untouched.
|
||||
* The markup it contains is what the client is expected to *re-typeset*; what the
|
||||
* server promises is the whole chapter and the paragraph boundaries within it,
|
||||
* and nothing about how it looks.
|
||||
*/
|
||||
export function renderChapterHtml(body: string): Buffer {
|
||||
const normalised = body.replace(LINE_ENDINGS, '\n');
|
||||
const lines = normalised.split('\n');
|
||||
// Only the *first* non-blank line is considered: a heading anywhere else is a
|
||||
// line of prose that happens to look like one, and promoting those would
|
||||
// restructure the book rather than render it.
|
||||
const firstIndex = lines.findIndex((line) => line.trim().length > 0);
|
||||
const hasHeading = firstIndex !== -1 && isHeading(lines[firstIndex] ?? '');
|
||||
const heading = hasHeading ? (lines[firstIndex] ?? '').trim() : '';
|
||||
const rest = hasHeading ? lines.slice(firstIndex + 1).join('\n') : normalised;
|
||||
|
||||
const parts = splitParagraphs(rest).map((paragraph) => `<p>${escapeHtml(paragraph)}</p>`);
|
||||
if (heading) parts.unshift(`<h3>${escapeHtml(heading)}</h3>`);
|
||||
const html = parts.join('\n');
|
||||
return Buffer.from(`<div class="txt-body">\n${html}\n</div>\n`, 'utf8');
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import {
|
||||
type TocEntry,
|
||||
} from './registry.ts';
|
||||
import { filenameMetadata } from '../metadata.ts';
|
||||
import { renderChapterHtml } from './text-html.ts';
|
||||
|
||||
/**
|
||||
* Plain text and Markdown.
|
||||
@@ -181,12 +180,14 @@ export const textHandler = registerFileHandler({
|
||||
seq: index,
|
||||
title: chapter.title,
|
||||
kind: 'chapter' as const,
|
||||
// The item *is* markup as far as a reader is concerned, and saying so here
|
||||
// is what stops the client from having to special-case the text format.
|
||||
// The reference stays `chapter:<n>` so a position saved before this field
|
||||
// The item is a *document* as far as a reader is concerned even though the
|
||||
// bytes are characters: the client turns it into one, and saying so here is
|
||||
// what stops the client from having to special-case the text format. The
|
||||
// reference stays `chapter:<n>` so a position saved before this field
|
||||
// existed still resolves; `format` tells the client which rendition to ask
|
||||
// for (see `ContentItem.format`).
|
||||
mediaType: 'text/html; charset=utf-8',
|
||||
// for (see `ContentItem.format`), and `html` now means "the whole chapter,
|
||||
// to be typeset" rather than "server-rendered markup".
|
||||
mediaType: 'text/plain; charset=utf-8',
|
||||
href: `chapter:${index}`,
|
||||
format: 'html' as const,
|
||||
})),
|
||||
@@ -206,40 +207,45 @@ export const textHandler = registerFileHandler({
|
||||
|
||||
/**
|
||||
* Body text. Three addressing modes, because clients need all three:
|
||||
* - `chapter-html:<n>` one whole chapter as a document (the reader draws this)
|
||||
* - `chapter:<n>` for the same chapter as plain text, for callers that want
|
||||
* the characters rather than a document (TTS, search, a diff)
|
||||
* - `chapter-html:<n>` one whole chapter, as the characters it is stored as
|
||||
* - `chapter:<n>` the same chapter, capped at `size` (a streaming window)
|
||||
* - `chunk:<byteOffset>` for streaming a novel with no headings
|
||||
*
|
||||
* Neither rendition carries the reader's typography. Indentation, paragraph
|
||||
* spacing and the removal of a scraper's leading spaces are per-device reading
|
||||
* preferences, applied by the client over whatever text either form delivers
|
||||
* (see `web/src/formats/segments.ts`); what the server owes the reader is the
|
||||
* chapter entire, and the paragraph boundaries it can only infer with the file
|
||||
* in hand.
|
||||
* Not one of them carries the reader's typography, and not one of them carries
|
||||
* paragraph boundaries either. `chapter-html:` used to render `<p>` markup on
|
||||
* the server; it no longer does, because paragraph splitting is a *reading*
|
||||
* decision (see `web/src/formats/segments.ts`) and doing it here meant the
|
||||
* windowed path got paragraphs while the streamed path could not, and meant the
|
||||
* reader's indent was a round trip. What the server owes the reader is the
|
||||
* chapter entire; the client typesets it.
|
||||
*
|
||||
* `chapter-html:` is therefore now an alias for the whole chapter as plain
|
||||
* characters, with the one difference that it is *not* capped: `chapter:` bounds
|
||||
* its reply because it is a window for streaming, and bounding a chapter the
|
||||
* reader is about to read would hand them a chapter that stops mid-sentence.
|
||||
*/
|
||||
async asset(ctx: HandlerContext, req): Promise<AssetPayload> {
|
||||
const size = Math.min(MAX_CHUNK_BYTES, DEFAULT_CHUNK_BYTES);
|
||||
const { text } = await readTextFile(ctx.absPath);
|
||||
|
||||
// The whole chapter, uncapped. See the method comment: `chapter:` bounds its
|
||||
// reply because it is a streaming window, and that bound must not be applied
|
||||
// to a chapter the reader is about to read in one piece.
|
||||
if (req.ref.startsWith(CHAPTER_HTML_REF)) {
|
||||
const index = Number.parseInt(req.ref.slice(CHAPTER_HTML_REF.length), 10);
|
||||
const { chapters, lines } = splitChapters(text);
|
||||
const chapter = chapters[index];
|
||||
if (!chapter) throw new Error(`chapter ${index} is out of range`);
|
||||
// Deliberately *not* capped at `size`, unlike `chapter:` below. The two
|
||||
// renditions answer different questions and the cap belongs to only one of
|
||||
// them: `chapter:` is a *window* into the file for streaming, so bounding it
|
||||
// is the feature; this is one whole chapter for a reader, so bounding it
|
||||
// would leave them with a chapter that stops mid-sentence. The client's own
|
||||
// `textToParagraphHtml` is what turns it into paragraphs — this is the
|
||||
// chapter's characters, complete, and the reader typesets them.
|
||||
const body = lines.slice(chapter.startLine, chapter.endLine + 1).join('\n');
|
||||
const data = renderChapterHtml(body);
|
||||
return {
|
||||
data,
|
||||
contentType: 'text/html; charset=utf-8',
|
||||
size: data.byteLength,
|
||||
data: Buffer.from(body, 'utf8'),
|
||||
// Plain characters, not `text/html`. The content type has to say what the
|
||||
// body is: served as HTML, a chapter whose text happens to contain `<`
|
||||
// would be parsed as markup by anything that trusted the label, and the
|
||||
// client's own escaping — the thing that makes a TXT safe — would be
|
||||
// sidestepped. The client typesets from this text; it does not trust it.
|
||||
contentType: 'text/plain; charset=utf-8',
|
||||
size: Buffer.byteLength(body, 'utf8'),
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -645,10 +645,14 @@ describe('reading endpoints per format', () => {
|
||||
assert.ok(!res.body.includes('风来了'), 'chapter content must not bleed into the next chapter');
|
||||
});
|
||||
|
||||
test('txt chapter content is also offered as renderable markup', async () => {
|
||||
// The rendition the reader actually draws. `chapter:` is the characters; this
|
||||
// is the same chapter as a document, which is what gives a TXT paragraphs it
|
||||
// can style instead of a wall of text with no line breaks in it.
|
||||
test('the chapter-html reference hands over the whole chapter, unmarked up', async () => {
|
||||
// This reference used to answer with server-rendered `<p>` markup. It no
|
||||
// longer does: paragraph boundaries are a *reading* decision, and rendering
|
||||
// them here meant the reader's indent was a round trip and a windowed chapter
|
||||
// looked different from a streamed one. What it answers with is the chapter's
|
||||
// characters and nothing else — the same bytes `chapter:` gives, minus the
|
||||
// streaming cap — which is what makes one typesetting pass on the client the
|
||||
// single definition of a paragraph.
|
||||
const book = await findBook('小说');
|
||||
const res = await app.inject({
|
||||
method: 'GET',
|
||||
@@ -656,11 +660,32 @@ describe('reading endpoints per format', () => {
|
||||
headers: auth(),
|
||||
});
|
||||
assert.equal(res.statusCode, 200, res.body);
|
||||
assert.match(res.headers['content-type'] as string, /text\/html/);
|
||||
assert.match(res.body, /<p>雨来了。<\/p>/);
|
||||
assert.match(res.headers['content-type'] as string, /text\/plain/);
|
||||
assert.match(res.body, /雨来了/);
|
||||
assert.doesNotMatch(res.body, /<p>/, 'the server must not typeset any more');
|
||||
assert.doesNotMatch(res.body, /<div/, 'the server must not wrap the chapter any more');
|
||||
assert.ok(!res.body.includes('风来了'), 'chapter content must not bleed into the next chapter');
|
||||
});
|
||||
|
||||
test('chapter-html is not capped the way the streaming chapter reference is', async () => {
|
||||
// `chapter:` is a *window* for streaming, so bounding its reply is the
|
||||
// feature. `chapter-html:` is a chapter the reader is about to read in one
|
||||
// piece, so bounding it would hand them a chapter that stops mid-sentence.
|
||||
const long = Array.from({ length: 4000 }, (_, i) => `第 ${i} 行的内容,足够长以越过流式窗口。`).join('\n');
|
||||
const target = join(booksDir, '超长章节.txt');
|
||||
await writeFile(target, `第一章 长\n${long}\n`, 'utf8');
|
||||
await ctx.scanner.scan();
|
||||
const book = await findBook('超长章节');
|
||||
const res = await app.inject({
|
||||
method: 'GET',
|
||||
url: `/api/v1/books/${book.id}/assets?ref=${encodeURIComponent('chapter-html:0')}`,
|
||||
headers: auth(),
|
||||
});
|
||||
assert.equal(res.statusCode, 200, res.body);
|
||||
const text = res.body;
|
||||
assert.ok(text.includes('第 3999 行'), 'the last line of the chapter must be present');
|
||||
});
|
||||
|
||||
test('the manifest advertises the markup rendition without moving the reference', async () => {
|
||||
// The reference is also the reading position's identity, so it has to stay
|
||||
// `chapter:<n>` across this change; `format` is what carries the new choice.
|
||||
|
||||
@@ -1,132 +0,0 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { test, describe } from 'node:test';
|
||||
import { escapeHtml, renderChapterHtml, splitParagraphs } from '../src/indexer/formats/text-html.ts';
|
||||
|
||||
/**
|
||||
* The paragraph split is where every interesting failure of the TXT rendition
|
||||
* lives, so it gets the tests. The escaping and the wrapping around it are
|
||||
* mechanical.
|
||||
*/
|
||||
describe('splitParagraphs', () => {
|
||||
test('a blank line separates paragraphs', () => {
|
||||
assert.deepEqual(splitParagraphs('第一段\n\n第二段'), ['第一段', '第二段']);
|
||||
});
|
||||
|
||||
test('a single newline inside a paragraph is a soft wrap, not a break', () => {
|
||||
// The common shape in a scraped novel: the source wraps a paragraph across
|
||||
// two lines. Splitting there would truncate the first half into its own
|
||||
// paragraph and indent a continuation.
|
||||
assert.deepEqual(splitParagraphs('这句被硬换行了\n但仍然是同一段'), ['这句被硬换行了但仍然是同一段']);
|
||||
});
|
||||
|
||||
test('a line-ending sentence starts a new paragraph without a blank line', () => {
|
||||
// The other common shape: every paragraph on its own line, no blank lines at
|
||||
// all. Without this rule such a file renders as one slab — which is exactly
|
||||
// the bug this rendition exists to fix.
|
||||
assert.deepEqual(
|
||||
splitParagraphs('他走了。\n她留下了。\n天亮了。'),
|
||||
['他走了。', '她留下了。', '天亮了。'],
|
||||
);
|
||||
});
|
||||
|
||||
test('does not split a continuation line that merely starts with punctuation', () => {
|
||||
// The lookahead requires the next line to start a sentence. A line beginning
|
||||
// with a comma, a closing quote or an ellipsis is a continuation of the one
|
||||
// above and must stay joined.
|
||||
assert.deepEqual(splitParagraphs('他走了。\n,她没有回头。'), ['他走了。,她没有回头。']);
|
||||
assert.deepEqual(splitParagraphs('他走了。\n」她说。'), ['他走了。」她说。']);
|
||||
});
|
||||
|
||||
test('handles CRLF and lone CR line endings', () => {
|
||||
// A lone CR is a line ending, not a character inside a word: treated as one,
|
||||
// a file off a serial console or an old conversion is a single paragraph.
|
||||
assert.deepEqual(splitParagraphs('一段\r\n\r\n二段'), ['一段', '二段']);
|
||||
assert.deepEqual(splitParagraphs('他走了。\r她留下了。'), ['他走了。', '她留下了。']);
|
||||
});
|
||||
|
||||
test('drops empty paragraphs rather than rendering blank blocks', () => {
|
||||
assert.deepEqual(splitParagraphs('\n\n\n\n一段\n\n\n'), ['一段']);
|
||||
assert.deepEqual(splitParagraphs(' \n\n '), []);
|
||||
});
|
||||
|
||||
test('an empty body produces no paragraphs', () => {
|
||||
assert.deepEqual(splitParagraphs(''), []);
|
||||
});
|
||||
});
|
||||
|
||||
describe('escapeHtml', () => {
|
||||
test('escapes the characters that would otherwise rewrite the page', () => {
|
||||
assert.equal(escapeHtml('<script>alert(1)</script>'), '<script>alert(1)</script>');
|
||||
assert.equal(escapeHtml('a & b'), 'a & b');
|
||||
assert.equal(escapeHtml(`"'`), '"'');
|
||||
});
|
||||
});
|
||||
|
||||
describe('renderChapterHtml', () => {
|
||||
test('wraps paragraphs in a .txt-body div so the client can scope its rules', () => {
|
||||
const html = renderChapterHtml('第一段\n\n第二段').toString('utf8');
|
||||
assert.match(html, /^<div class="txt-body">\n/);
|
||||
assert.equal(html.match(/<p>/g)?.length, 2);
|
||||
assert.match(html, /<p>第一段<\/p>/);
|
||||
assert.match(html, /<p>第二段<\/p>/);
|
||||
});
|
||||
|
||||
test('carries no typography, so the readers own settings are what apply', () => {
|
||||
// The indent, the paragraph spacing and the removal of a scraper's leading
|
||||
// spaces are per-device reading preferences, and a server that baked one of
|
||||
// them into the payload is what made them a round trip — and what made a book
|
||||
// read windowed look different from the same book read whole. The assertion is
|
||||
// on the absence, because absence is the contract.
|
||||
const html = renderChapterHtml('一段').toString('utf8');
|
||||
assert.doesNotMatch(html, /data-indent/);
|
||||
assert.doesNotMatch(html, /style=/);
|
||||
assert.doesNotMatch(html, /text-indent/);
|
||||
});
|
||||
|
||||
test('a file containing markup cannot inject elements', () => {
|
||||
const html = renderChapterHtml(['<img src=x onerror=alert(1)>', '', '<b>粗</b>'].join('\n')).toString('utf8');
|
||||
// The characters survive as *text*; what must not survive is an element or an
|
||||
// attribute, so the assertion is on the markup rather than on the words — an
|
||||
// escaped `onerror` is a paragraph, not a handler.
|
||||
assert.doesNotMatch(html, /<img/);
|
||||
assert.doesNotMatch(html, /<b>/);
|
||||
assert.match(html, /<img src=x onerror=alert\(1\)>/);
|
||||
assert.match(html, /<b>粗<\/b>/);
|
||||
});
|
||||
|
||||
test('a chapter is rendered whole, with no truncation', () => {
|
||||
// `chapter:` sliced the body at 256KB, which cut a long chapter in half.
|
||||
// Stated as a test so the cap cannot come back by accident.
|
||||
// Comfortably past the 256KB *byte* budget the plain-text path used: Chinese
|
||||
// is three bytes per character, so the character count that overflowed it is
|
||||
// under 90000. A chapter this size is unusual but real — web novels ship them.
|
||||
const body = Array.from({ length: 40_000 }, (_, i) => `第 ${i} 段的内容。`).join('\n');
|
||||
assert.ok(body.length > 256 * 1024 / 3, 'fixture must exceed the old character budget');
|
||||
const html = renderChapterHtml(body).toString('utf8');
|
||||
assert.match(html, /<p>第 39999 段的内容。<\/p>/);
|
||||
});
|
||||
|
||||
test('promotes the chapter heading instead of gluing it to the first sentence', () => {
|
||||
// `splitChapters` puts the heading in the chapter's own slice, so without this
|
||||
// the body reads `第二章 落雨雨来了。` — the title and the first line as one
|
||||
// paragraph, which is what a reader reports as "格式乱了".
|
||||
const html = renderChapterHtml(['第二章 落雨', '雨来了。'].join('\n')).toString('utf8');
|
||||
assert.match(html, /<h3>第二章 落雨<\/h3>/);
|
||||
assert.match(html, /<p>雨来了。<\/p>/);
|
||||
assert.doesNotMatch(html, /落雨雨来了/);
|
||||
});
|
||||
|
||||
test('leaves prose that merely looks like a heading where it is', () => {
|
||||
// Only the first line is considered, and only when it is short enough to be a
|
||||
// title: a mid-chapter line that mentions a chapter number is a sentence.
|
||||
const html = renderChapterHtml(['雨来了。', '他说第二章他看过。'].join('\n')).toString('utf8');
|
||||
assert.doesNotMatch(html, /<h3>/);
|
||||
assert.match(html, /<p>他说第二章他看过。<\/p>/);
|
||||
});
|
||||
|
||||
test('returns utf-8 bytes whose length matches the payload it reports', () => {
|
||||
const buffer = renderChapterHtml('中文内容\n\n第二段');
|
||||
assert.equal(buffer.toString('utf8').length <= buffer.byteLength, true);
|
||||
assert.equal(buffer.byteLength, Buffer.byteLength(buffer.toString('utf8'), 'utf8'));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,20 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { test, describe } from 'node:test';
|
||||
import { escapeHtml } from '../src/indexer/formats/text-html.ts';
|
||||
|
||||
/**
|
||||
* The escaping helper is all that is left of the server's TXT rendition.
|
||||
*
|
||||
* The paragraph split that used to be tested here moved to the client
|
||||
* (`web/src/formats/segments.ts`, tested in `web/test/segments.test.ts`), because
|
||||
* deciding where a paragraph is is a *reading* decision and not a property of the
|
||||
* file. What is left on the server is the one mechanical thing: making a chapter's
|
||||
* characters safe to put in a page, for the paths that still need it.
|
||||
*/
|
||||
describe('escapeHtml', () => {
|
||||
test('escapes the characters that would otherwise rewrite the page', () => {
|
||||
assert.equal(escapeHtml('<script>alert(1)</script>'), '<script>alert(1)</script>');
|
||||
assert.equal(escapeHtml('a & b'), 'a & b');
|
||||
assert.equal(escapeHtml(`"'`), '"'');
|
||||
});
|
||||
});
|
||||
+73
-8
@@ -1,5 +1,6 @@
|
||||
import { ApiError } from '../api/errors.ts';
|
||||
import type { ReaderApi, NoteInput } from '../api/client.ts';
|
||||
import type { SyncPull } from '../api/types.ts';
|
||||
import { isOutboxEmpty, type OfflineStore } from '../store/offline.ts';
|
||||
import type { Connectivity, Platform } from './platform.ts';
|
||||
|
||||
@@ -93,7 +94,24 @@ export class SyncEngine {
|
||||
for (const listener of this.listeners) listener(status);
|
||||
}
|
||||
|
||||
/**
|
||||
* Publishes a state change, and *only* a change.
|
||||
*
|
||||
* The guard is the fix for a re-render storm. The reader screen's status
|
||||
* listener repaints the whole chrome tree on every notification, and a sync
|
||||
* cycle announces itself twice — `syncing` on the way in, `idle` on the way out —
|
||||
* so a reader flipping through a book repainted that tree three times per cycle
|
||||
* (the explicit `emit()` after `setState('idle')` was a third) whether or not
|
||||
* anything had actually changed. `syncNow` also runs on a 30s poll, so the storm
|
||||
* continued while the reader was doing nothing at all.
|
||||
*
|
||||
* A state or message that is already published carries no new information, and
|
||||
* the readers of this stream are *painters*: telling them again is pure cost.
|
||||
* Deduplicating here rather than at each listener means every listener gets the
|
||||
* same guarantee, and a listener added later cannot reintroduce the storm.
|
||||
*/
|
||||
private setState(state: SyncState, message = ''): void {
|
||||
if (this.state === state && this.message === message) return;
|
||||
this.state = state;
|
||||
this.message = message;
|
||||
this.emit();
|
||||
@@ -156,7 +174,21 @@ export class SyncEngine {
|
||||
return this.requested !== null;
|
||||
}
|
||||
|
||||
private scheduleNext(delay: number): void {
|
||||
/**
|
||||
* Arms the next automatic poll.
|
||||
*
|
||||
* A timer that is already armed is *kept* rather than re-armed, and the
|
||||
* difference is a real one: `syncNow` runs on demand (every `schedule()` push)
|
||||
* as well as on the poll, and resetting the timer on each of those meant the
|
||||
* poll was pushed out by every page turn. On a book being read, the 30-second
|
||||
* poll then never fired at all — the pull half of sync stopped happening while
|
||||
* the push half ran constantly, which is the asymmetry a reader sees as "怎么还
|
||||
* 在一直请求" and as a shelf that never picks up another device's progress.
|
||||
*/
|
||||
private scheduleNext(delay: number, force = false): void {
|
||||
// A backoff retry *replaces* the poll rather than waiting behind it: the poll
|
||||
// is 30 seconds away, and a failure that retries in 2 has to be allowed to.
|
||||
if (this.timer && !force) return;
|
||||
if (this.timer) clearTimeout(this.timer);
|
||||
this.timer = setTimeout(() => {
|
||||
this.timer = null;
|
||||
@@ -172,11 +204,24 @@ export class SyncEngine {
|
||||
* the same time.
|
||||
*/
|
||||
async syncNow(): Promise<void> {
|
||||
if (this.running) return;
|
||||
if (this.running) {
|
||||
// A run is already in flight and will arm the next poll itself. Re-arming
|
||||
// here would be a second timer, and the loser would fire a duplicate cycle.
|
||||
return;
|
||||
}
|
||||
if (!this.api.currentSession()) {
|
||||
this.setState('signed-out');
|
||||
return;
|
||||
}
|
||||
// A push that has been asked for and not yet run is *this* work, a few
|
||||
// milliseconds early. Letting the poll run first would send the same outbox
|
||||
// twice and pull twice, and the second answer would be identical to the first
|
||||
// — which is exactly the "sync 接口在不停重复" a reader sees in the network
|
||||
// panel. The pending push will do the job; the poll re-arms itself below.
|
||||
if (this.requested) {
|
||||
this.scheduleNext(IDLE_INTERVAL_MS);
|
||||
return;
|
||||
}
|
||||
this.running = true;
|
||||
this.setState('syncing');
|
||||
try {
|
||||
@@ -187,8 +232,19 @@ export class SyncEngine {
|
||||
return;
|
||||
}
|
||||
|
||||
await this.pushOutbox();
|
||||
const pull = await this.api.pull(this.offline.current.serverTime);
|
||||
// One round trip, not two.
|
||||
//
|
||||
// `POST /api/v1/sync` answers with the merged state — the server's own
|
||||
// comment says so: "Returning the merged state lets the client reconcile in
|
||||
// the same round trip instead of issuing a follow-up pull." A push used to
|
||||
// throw that answer away and then issue a *second* request for it, so every
|
||||
// page turn cost a POST and a GET carrying the same body. Taking the push's
|
||||
// answer is what makes a page turn cost one request.
|
||||
//
|
||||
// A push that had nothing to send (an idle poll) returns null, and then the
|
||||
// pull is the real work — that is the poll's whole purpose.
|
||||
const pushed = await this.pushOutbox();
|
||||
const pull = pushed ?? (await this.api.pull(this.offline.current.serverTime));
|
||||
await this.offline.applyPulled(pull);
|
||||
this.rememberNoteBooks();
|
||||
|
||||
@@ -196,7 +252,6 @@ export class SyncEngine {
|
||||
// A rejection message has to survive the state transition to idle,
|
||||
// otherwise the reader never learns that a record was dropped.
|
||||
this.setState('idle', this.rejectedCount > 0 ? `${this.rejectedCount} 条记录被服务端拒绝` : this.message);
|
||||
this.emit();
|
||||
this.scheduleNext(IDLE_INTERVAL_MS);
|
||||
} catch (err) {
|
||||
this.handleFailure(err);
|
||||
@@ -213,7 +268,7 @@ export class SyncEngine {
|
||||
* the record's own update will still be dirty after this returns, so the next
|
||||
* cycle picks it up instead of it being cleared by a response that predates it.
|
||||
*/
|
||||
private async pushOutbox(): Promise<void> {
|
||||
private async pushOutbox(): Promise<SyncPull | null> {
|
||||
const batch = this.offline.outbox();
|
||||
const deletes: NoteInput[] = batch.deletes.map((id) => ({
|
||||
id,
|
||||
@@ -244,7 +299,8 @@ export class SyncEngine {
|
||||
})),
|
||||
...deletes,
|
||||
];
|
||||
if (batch.progress.length === 0 && notes.length === 0) return;
|
||||
// Nothing to send: this is an idle poll, and the caller does the pull.
|
||||
if (batch.progress.length === 0 && notes.length === 0) return null;
|
||||
|
||||
// The server caps a batch at 5000 records; chunk well below that so a long
|
||||
// offline session cannot fail wholesale on one oversized request.
|
||||
@@ -252,6 +308,10 @@ export class SyncEngine {
|
||||
const batches = Math.max(1, Math.ceil(Math.max(batch.progress.length, notes.length) / CHUNK));
|
||||
let serverTime = 0;
|
||||
let rejected = 0;
|
||||
// The last chunk's answer is the merged state as of the final write, which is
|
||||
// the only one worth reconciling against: an earlier chunk's `progress` list
|
||||
// predates the writes that came after it.
|
||||
let merged: SyncPull | null = null;
|
||||
for (let index = 0; index < batches; index += 1) {
|
||||
const offset = index * CHUNK;
|
||||
const progressChunk = batch.progress.slice(offset, offset + CHUNK);
|
||||
@@ -262,6 +322,7 @@ export class SyncEngine {
|
||||
});
|
||||
serverTime = Math.max(serverTime, result.serverTime);
|
||||
rejected += result.rejected;
|
||||
merged = result;
|
||||
}
|
||||
await this.offline.markDelivered(batch, serverTime);
|
||||
// Rejections are informational. The server refuses records it can never
|
||||
@@ -270,6 +331,7 @@ export class SyncEngine {
|
||||
// on the engine rather than in a transient message so the status bar can
|
||||
// still show it after the state settles.
|
||||
this.rejectedCount = rejected;
|
||||
return merged;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -302,7 +364,10 @@ export class SyncEngine {
|
||||
}
|
||||
this.setState('error', err instanceof Error ? err.message : '同步失败');
|
||||
this.backoff = Math.min(this.backoff * 2, RETRY_MAX_MS);
|
||||
this.scheduleNext(this.backoff);
|
||||
// `force`: an error's exponential backoff must be able to shorten a wait the
|
||||
// poll has already armed, or the first retry after a blip would be 30 seconds
|
||||
// late and the backoff would appear not to work.
|
||||
this.scheduleNext(this.backoff, true);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -190,17 +190,71 @@ export function escapeHtml(text: string): string {
|
||||
.replace(/'/g, ''');
|
||||
}
|
||||
|
||||
/**
|
||||
* A chapter heading, promoted out of the body.
|
||||
*
|
||||
* A chapter's own slice *includes* its heading line — chapter splitting records
|
||||
* the heading as the chapter's first line, which is what makes a saved position
|
||||
* land on the title rather than one line into the chapter. Left as ordinary prose
|
||||
* it renders as `第二章 落雨雨来了。`: the title and the first sentence glued
|
||||
* together with no separation, which is what a reader reports as "格式乱了".
|
||||
*
|
||||
* Promoted rather than merely split off, because it is the one piece of structure
|
||||
* a TXT actually has, and a reader scanning for a chapter boundary is looking for
|
||||
* exactly this. The patterns are deliberately narrow for the same reason the
|
||||
* chapter *split* is: a false positive restructures the book rather than
|
||||
* rendering it.
|
||||
*/
|
||||
const HEADING_PATTERNS: readonly RegExp[] = [
|
||||
/^第\s*[0-90-9零一二三四五六七八九十百千万两]+\s*[章回节卷部篇集]\s*[::.、\-—]?\s*.{0,60}$/,
|
||||
/^(?:序章|序言|楔子|引子|前言|后记|尾声|终章|番外|附录)\s*.{0,40}$/,
|
||||
/^[卷部]\s*[0-90-9零一二三四五六七八九十百千万两]+\s*.{0,60}$/,
|
||||
/^Chapter\s*\d+.*$/i,
|
||||
/^#{1,3}\s+.+$/,
|
||||
];
|
||||
|
||||
/** The longest line still plausibly a heading rather than prose. */
|
||||
const MAX_HEADING_LENGTH = 60;
|
||||
|
||||
function isHeadingLine(line: string): boolean {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.length === 0 || trimmed.length > MAX_HEADING_LENGTH) return false;
|
||||
return HEADING_PATTERNS.some((pattern) => pattern.test(trimmed));
|
||||
}
|
||||
|
||||
/**
|
||||
* Splits a chapter body into its heading line and the rest.
|
||||
*
|
||||
* Only the *first* non-blank line is considered: a heading anywhere else is a line
|
||||
* of prose that happens to look like one, and promoting those would restructure
|
||||
* the book rather than render it.
|
||||
*/
|
||||
export function splitChapterHeading(text: string): { heading: string; body: string } {
|
||||
const normalised = text.replace(LINE_ENDINGS, '\n');
|
||||
const lines = normalised.split('\n');
|
||||
const firstIndex = lines.findIndex((line) => line.trim().length > 0);
|
||||
if (firstIndex === -1) return { heading: '', body: '' };
|
||||
if (!isHeadingLine(lines[firstIndex] ?? '')) return { heading: '', body: normalised };
|
||||
return {
|
||||
heading: (lines[firstIndex] ?? '').trim(),
|
||||
body: lines.slice(firstIndex + 1).join('\n'),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The same rendition, wrapped so the client's scoping class is present.
|
||||
*
|
||||
* The string form used by the loaders, and the one thing it adds over
|
||||
* `textToParagraphHtml` is the wrapper: a chapter's markup is the unit that
|
||||
* The string form used by the loaders, and what it adds over
|
||||
* `textToParagraphHtml` is the wrapper — a chapter's markup is the unit that
|
||||
* reaches a shadow root, and the class has to travel with it rather than being
|
||||
* added by whoever renders it — a server-rendered chapter carries it in its own
|
||||
* response, so a locally loaded one must not be the odd one out.
|
||||
* added by whoever renders it — plus the heading promotion, which is the same
|
||||
* decision on every path (a locale's own file, the windowed manifest, the stream)
|
||||
* rather than one the server made for two of them.
|
||||
*/
|
||||
export function textToChapterHtml(text: string): string {
|
||||
return `<div class="txt-body">\n${textToParagraphHtml(text)}\n</div>\n`;
|
||||
const { heading, body } = splitChapterHeading(text);
|
||||
const promoted = heading ? `<h3>${escapeHtml(heading)}</h3>\n` : '';
|
||||
return `<div class="txt-body">\n${promoted}${textToParagraphHtml(body)}\n</div>\n`;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -45,6 +45,19 @@ export interface Section {
|
||||
* to express later without changing the contract.
|
||||
*/
|
||||
render?: RenderMode;
|
||||
/**
|
||||
* The section's body is plain characters to be typeset, not authored markup.
|
||||
*
|
||||
* Carried per section because the *manifest* is where the declaration lives (a
|
||||
* content item's `format: 'html'`), and because inferring it from the body is a
|
||||
* guess that fails in exactly the case this exists for: the server used to wrap a
|
||||
* TXT chapter in `<div class="txt-body">`, and now it sends bare characters, so a
|
||||
* marker-sniffing check answers "not plain text" for a chapter that is nothing
|
||||
* *but* plain text — and the reader then draws a novel with no paragraphs, no
|
||||
* indent and no stylesheet applied, which is a defect that looks like a typography
|
||||
* preference rather than a bug.
|
||||
*/
|
||||
plainText?: boolean;
|
||||
/**
|
||||
* Path this section's bytes can be fetched from, when the section is one
|
||||
* addressable file. This is what lets a native host fetch a page itself
|
||||
|
||||
@@ -95,6 +95,11 @@ export function createStagedDoc(options: StagedDocOptions): StagedDoc {
|
||||
depth: 0,
|
||||
render: item.kind === 'page' ? 'image' : 'reflowable',
|
||||
path: item.href,
|
||||
// The manifest's own declaration, not a guess from the body. A content item that
|
||||
// says `format: 'html'` is saying "these are characters for you to typeset", and
|
||||
// it is the only party that knows: the body no longer carries a marker to sniff
|
||||
// (see `Section.plainText`).
|
||||
...(item.format === 'html' ? { plainText: true } : {}),
|
||||
}));
|
||||
|
||||
const base: BookDoc = {
|
||||
@@ -157,6 +162,9 @@ export function createStagedDoc(options: StagedDocOptions): StagedDoc {
|
||||
depth: 0,
|
||||
render: item.kind === 'page' ? 'image' : 'reflowable',
|
||||
path: item.href,
|
||||
// Same declaration as above, and carried across a window swap for the same
|
||||
// reason: a jump to chapter 900 must not turn the book into an EPUB.
|
||||
...(item.format === 'html' ? { plainText: true } : {}),
|
||||
});
|
||||
}
|
||||
return local;
|
||||
|
||||
@@ -1671,9 +1671,14 @@ function toArrayBuffer(bytes: Uint8Array): ArrayBuffer {
|
||||
* chapter one. Prefixing on the client keeps old positions resolving and new
|
||||
* chapters rendering.
|
||||
*
|
||||
* `chapter-html:` is the server's markup rendition of `chapter:`. An item that does
|
||||
* not declare `html` is fetched exactly as the server named it, which is the
|
||||
* behaviour every other format already relies on.
|
||||
* `chapter-html:` is `chapter:` without the streaming cap — one whole chapter for a
|
||||
* reader who is about to read it in one piece. It used to mean "server-rendered
|
||||
* markup" as well; that half is gone (the client typesets, see
|
||||
* `web/src/formats/segments.ts`) and the reference name is kept because it is the
|
||||
* identity a cached section and a saved position are keyed on, and because a
|
||||
* client that asks for it must keep working. An item that does not declare `html`
|
||||
* is fetched exactly as the server named it, which is the behaviour every other
|
||||
* format already relies on.
|
||||
*/
|
||||
function renditionRef(item: { href: string; format?: string }): string {
|
||||
if (item.format !== 'html') return item.href;
|
||||
|
||||
+77
-38
@@ -9,7 +9,7 @@ import {
|
||||
sanitiseInjectedContent,
|
||||
type BookShadowHost,
|
||||
} from './shadow.ts';
|
||||
import { escapeHtml, textToParagraphHtml } from '../formats/segments.ts';
|
||||
import { escapeHtml, textToChapterHtml } from '../formats/segments.ts';
|
||||
import { bookPercentage, formatLocator, parseLocator } from './locator.ts';
|
||||
import { collectSpokenChunks, type SpokenChunk } from '../render/tts-text.ts';
|
||||
import { adoptWindow } from '../formats/windowed.ts';
|
||||
@@ -525,63 +525,102 @@ export class ReaderView {
|
||||
* indent control beside them.
|
||||
*/
|
||||
isPlainText(section: Section = this.doc.sections[this.sectionIndex] ?? EMPTY_SECTION): boolean {
|
||||
if (this.doc.format === 'txt') return true;
|
||||
// The declaration wins outright. `format: 'txt'` is the book saying so and
|
||||
// `plainText` is the *section* saying so, which is the one that can be right when
|
||||
// a manifest windows a TXT as `reflowable` — a reasonable choice, since a TXT is
|
||||
// reflowable, and one that made the chapter-level answer differ from the
|
||||
// book-level one.
|
||||
if (this.doc.format === 'txt' || section.plainText) return true;
|
||||
// The marker is kept as the last resort, for a section that reached here from an
|
||||
// older server (or a cache written by one) which *did* wrap its chapters in
|
||||
// `<div class="txt-body">`. It is deliberately no longer the primary signal: a
|
||||
// server that sends bare characters carries no marker, so sniffing for one
|
||||
// answers "not plain text" for a chapter that is nothing but plain text.
|
||||
const html = section.html ?? '';
|
||||
return /<div[^>]*class="[^"]*\btxt-body\b/.test(html);
|
||||
}
|
||||
|
||||
/**
|
||||
* Re-renders a chapter of a plain-text book through the reader's own split.
|
||||
* Renders a chapter of a plain-text book through the reader's own split.
|
||||
*
|
||||
* Two sources reach this method and they need the same treatment:
|
||||
* The input is the *characters* of the chapter, however they arrived:
|
||||
*
|
||||
* - the server's `chapter-html:<n>` rendition (see `text-html.ts`), which is
|
||||
* that book's characters as markup; and
|
||||
* - a `chapter:<n>` body fetched by the windowed path, which is the book's
|
||||
* characters and nothing else.
|
||||
* - a `chapter-html:<n>` body fetched by the windowed path, which is that
|
||||
* book's text and nothing else (the server stopped typesetting — see
|
||||
* `server/src/indexer/formats/text-html.ts`); or
|
||||
* - a body from a server that still renders markup, which is handled by
|
||||
* reading its blocks back as text rather than by trusting its elements.
|
||||
*
|
||||
* Both end up here because *both* are the reader's own rendering of a file that
|
||||
* has none, and the thing the reader asked for — a per-device indent, paragraph
|
||||
* spacing, the removal of a scraper's leading spaces — is a property of the
|
||||
* rendering, not of the file. Doing it server-side meant the settings could only
|
||||
* apply to the rendition the server had already produced, and a book read
|
||||
* windowed got no paragraphs at all; doing it here means one code path and no
|
||||
* request, and the server's job shrinks to handing over the text.
|
||||
* apply to the rendition the server had already produced, and it meant the two
|
||||
* transports could disagree about where a paragraph was.
|
||||
*
|
||||
* Guarded by `isPlainText`, and the guard is why an EPUB is safe: an ordinary
|
||||
* novel never produces a `txt-body` wrapper, so it goes through untouched (the
|
||||
* novel never carries the `txt-body` marker, so it goes through untouched (the
|
||||
* one thing in this file that must not happen is a re-typesetting pass that runs
|
||||
* on every format, because that would mangle every EPUB).
|
||||
*/
|
||||
private retypePlainText(section: Section): string {
|
||||
const raw = section.html ?? '';
|
||||
if (!this.isPlainText(section)) return raw;
|
||||
const container = document.createElement('div');
|
||||
container.innerHTML = extractBody(raw);
|
||||
const wrapper = container.querySelector('.txt-body');
|
||||
if (!wrapper) return raw;
|
||||
// A heading the server promoted out of the body is kept, and kept *first*:
|
||||
// it is the one piece of structure a TXT has, and re-deriving it here would
|
||||
// duplicate a heuristic the server already runs against the undecoded file.
|
||||
const heading = wrapper.querySelector('h3, h4');
|
||||
// The text is read back *per block*, and that is the whole job. Using
|
||||
// `wrapper.textContent` would concatenate the paragraphs into one run with no
|
||||
// separator, so a chapter that arrived already split would be re-split as a
|
||||
// single paragraph and every boundary it had would be lost — the exact defect
|
||||
// this method exists to remove, arriving from the other direction. Joining the
|
||||
// blocks with a blank line puts back the separator the split reads as a
|
||||
// boundary, and the boundaries *inside* a block are still inferred from its
|
||||
// text, which is what makes this work for a chapter the server sent as one
|
||||
// slab with no elements in it at all.
|
||||
const blocks = [...wrapper.children]
|
||||
.filter((element) => !/^H[1-6]$/.test(element.tagName))
|
||||
.map((element) => element.textContent ?? '')
|
||||
.filter((text) => text.trim().length > 0);
|
||||
const body = blocks.length > 0
|
||||
? blocks.join('\n\n')
|
||||
: wrapper.textContent ?? '';
|
||||
const promoted = heading ? `<h3>${escapeHtml(heading.textContent ?? '')}</h3>\n` : '';
|
||||
return `<div class="txt-body">\n${promoted}${textToParagraphHtml(body)}\n</div>\n`;
|
||||
// A `txt-body` wrapper means the body arrived as markup (this build no longer
|
||||
// produces one, but a window carried over from an older server or a cached
|
||||
// section might). Its blocks are read back *per block* and re-joined with a
|
||||
// blank line: `textContent` on the wrapper would concatenate them into one
|
||||
// run, and the re-split below would then see a single paragraph and destroy
|
||||
// every boundary the block structure was carrying.
|
||||
let text = raw;
|
||||
// A heading an older server promoted out of the body. It is kept, and kept
|
||||
// *first*, because it is the one piece of structure a TXT has and because
|
||||
// re-deriving it is a heuristic — running a heuristic twice is how two copies of
|
||||
// it come to disagree. Re-emitted below rather than fed back through the regex.
|
||||
let keptHeading = '';
|
||||
// Only a body that *is* markup is parsed as markup: a section the manifest
|
||||
// declared as plain text is characters all the way down, and running an HTML
|
||||
// parser over a novel whose text happens to contain `<` is how a paragraph gets
|
||||
// silently eaten.
|
||||
if (!section.plainText && /<div[^>]*class="[^"]*\btxt-body\b/.test(raw)) {
|
||||
const container = document.createElement('div');
|
||||
container.innerHTML = raw;
|
||||
const wrapper = container.querySelector('.txt-body');
|
||||
if (wrapper) {
|
||||
const heading = wrapper.querySelector('h3, h4');
|
||||
keptHeading = heading?.textContent?.trim() ?? '';
|
||||
// Read back *per block*, not with `textContent` on the wrapper: the latter
|
||||
// concatenates the paragraphs into one run, and the re-split below would
|
||||
// then see a single paragraph and destroy every boundary the block
|
||||
// structure was carrying.
|
||||
const blocks = [...wrapper.children]
|
||||
.filter((element) => !/^H[1-6]$/.test(element.tagName))
|
||||
.map((element) => element.textContent ?? '')
|
||||
.filter((block) => block.trim().length > 0);
|
||||
text = blocks.length > 0 ? blocks.join('\n\n') : wrapper.textContent ?? '';
|
||||
}
|
||||
} else {
|
||||
// Not markup: the chapter's characters, as fetched, and nothing else. This
|
||||
// is what the server sends now (see `server/src/indexer/formats/text.ts`).
|
||||
// `extractBody` is still applied because a plain-text body can be wrapped in
|
||||
// an `<html>`/`<body>` shell by a proxy, and unwrapping that is cheaper than
|
||||
// discovering it as a stray `<html>` inside the first paragraph.
|
||||
text = extractBody(raw);
|
||||
}
|
||||
// The heading is re-derived from the text like every other boundary, by
|
||||
// `segments.ts`, when the body did not already carry one: the chapter's first
|
||||
// line is its title (that is what `splitChapters` records and why a locator
|
||||
// lands on the title). A heading the body *did* carry is re-emitted directly
|
||||
// rather than fed back through the regex — it was already promoted once, and
|
||||
// a second pass would only be a chance to disagree with the first.
|
||||
const rendered = textToChapterHtml(text);
|
||||
if (keptHeading) {
|
||||
return rendered.replace(
|
||||
'<div class="txt-body">\n',
|
||||
`<div class="txt-body">\n<h3>${escapeHtml(keptHeading)}</h3>\n`,
|
||||
);
|
||||
}
|
||||
return rendered;
|
||||
}
|
||||
|
||||
private async renderReflowable(section: Section): Promise<void> {
|
||||
|
||||
@@ -514,15 +514,17 @@ describe('plain-text chapters are typeset by the reader', () => {
|
||||
return host.shadow;
|
||||
}
|
||||
|
||||
const serverChapter = [
|
||||
'<div class="txt-body">',
|
||||
'<h3>第二章 落雨</h3>',
|
||||
'<p> 雨来了。</p>',
|
||||
'<p>他走在雨里。</p>',
|
||||
'</div>',
|
||||
].join('\n');
|
||||
/**
|
||||
* A chapter exactly as the server now sends it: the characters, and nothing else.
|
||||
*
|
||||
* `chapter-html:<n>` used to answer with server-rendered `<p>` markup. It answers
|
||||
* with the text now, because paragraph boundaries are a *reading* decision — see
|
||||
* `server/src/indexer/formats/text.ts`. Every assertion below is written against
|
||||
* this shape, because this is the shape the reader actually receives.
|
||||
*/
|
||||
const serverChapter = ['第二章 落雨', ' 雨来了。', '他走在雨里。'].join('\n');
|
||||
|
||||
it('re-splits a serverside chapter and drops the leading spaces it carried', async () => {
|
||||
it('typesets the text the server sends, and drops the leading spaces it carried', async () => {
|
||||
// The whole point of moving the typography to the client: the server hands over
|
||||
// characters, and one of the characters a scraper wrote is the full-width space
|
||||
// that was standing in for an indent. Left in, it is indent added to whatever the
|
||||
@@ -535,17 +537,58 @@ describe('plain-text chapters are typeset by the reader', () => {
|
||||
expect(body).not.toBeNull();
|
||||
const paragraphs = [...body!.querySelectorAll('p')].map((p) => p.textContent);
|
||||
expect(paragraphs).toEqual(['雨来了。', '他走在雨里。']);
|
||||
// The heading the server promoted is kept, and stays first.
|
||||
// The chapter's first line is its title, and the *client* promotes it now —
|
||||
// otherwise it renders as `第二章 落雨雨来了。`, the title and the first sentence
|
||||
// glued together, which is what a reader reports as "格式乱了".
|
||||
expect(body!.querySelector('h3')?.textContent).toBe('第二章 落雨');
|
||||
});
|
||||
|
||||
it('escapes a chapter whose own text contains markup', async () => {
|
||||
// The server sends `text/plain` now and for exactly this reason: served as HTML,
|
||||
// a chapter containing `<b>` would be parsed as markup by anything that trusted
|
||||
// the label, and the escaping that makes a TXT safe would be sidestepped. The
|
||||
// assertion is on the *absence* of an element, not on the presence of the words.
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const view = make(container, txtDoc('<img src=x onerror=alert(1)>\n\n<b>粗</b>'));
|
||||
await view.open(0, 0);
|
||||
const shadow = shadowOf(container);
|
||||
expect(shadow.querySelector('.txt-body img')).toBeNull();
|
||||
expect(shadow.querySelector('.txt-body b')).toBeNull();
|
||||
const paragraphs = [...shadow.querySelectorAll('.txt-body > p')].map((p) => p.textContent);
|
||||
expect(paragraphs).toEqual(['<img src=x onerror=alert(1)>', '<b>粗</b>']);
|
||||
});
|
||||
|
||||
it('re-typesets a chapter an older server sent as markup', async () => {
|
||||
// A window may have been cached against a server that still rendered `<p>`s (or
|
||||
// a proxy may re-wrap one). The blocks are read back as text and re-split, so an
|
||||
// old response cannot be rendered as if it were a new one — and the heading it
|
||||
// carried is kept rather than re-derived, because a heuristic run twice is two
|
||||
// copies of the heuristic.
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const legacy = [
|
||||
'<div class="txt-body">',
|
||||
'<h3>第二章 落雨</h3>',
|
||||
'<p> 雨来了。</p>',
|
||||
'<p>他走在雨里。</p>',
|
||||
'</div>',
|
||||
].join('\n');
|
||||
const view = make(container, txtDoc(legacy));
|
||||
await view.open(0, 0);
|
||||
const body = shadowOf(container).querySelector('.txt-body');
|
||||
const paragraphs = [...body!.querySelectorAll('p')].map((p) => p.textContent);
|
||||
expect(paragraphs).toEqual(['雨来了。', '他走在雨里。']);
|
||||
expect(body!.querySelector('h3')?.textContent).toBe('第二章 落雨');
|
||||
});
|
||||
|
||||
it('adds paragraphs to a chapter the server sent as one slab', async () => {
|
||||
// A windowed TXT arrives through `chapter:<n>`, which is the characters with no
|
||||
// markup at all. Without this the reader gets a wall of text while the same book
|
||||
// read whole gets paragraphs — one book, two appearances, decided by transport.
|
||||
// A TXT whose chapters have no blank lines between paragraphs is the common
|
||||
// scraped shape. Without this the reader gets a wall of text — one book, two
|
||||
// appearances, decided by how the file happened to be written.
|
||||
const container = document.createElement('div');
|
||||
document.body.append(container);
|
||||
const view = make(container, txtDoc('<div class="txt-body">他走了。\n她留下了。\n天亮了。</div>'));
|
||||
const view = make(container, txtDoc('他走了。\n她留下了。\n天亮了。'));
|
||||
await view.open(0, 0);
|
||||
const paragraphs = [...shadowOf(container).querySelectorAll('.txt-body > p')].map((p) => p.textContent);
|
||||
expect(paragraphs).toEqual(['他走了。', '她留下了。', '天亮了。']);
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
endsAtParagraphBreak,
|
||||
escapeHtml,
|
||||
paragraphElement,
|
||||
splitChapterHeading,
|
||||
splitTextParagraphs,
|
||||
textToChapterHtml,
|
||||
textToParagraphHtml,
|
||||
@@ -86,6 +87,26 @@ describe('the markup form', () => {
|
||||
expect(textToChapterHtml('一段')).toMatch(/^<div class="txt-body">\n<p>一段<\/p>\n<\/div>\n$/);
|
||||
});
|
||||
|
||||
it('promotes the chapter title instead of gluing it to the first sentence', () => {
|
||||
// Chapter splitting records the heading as the chapter's *first line* — that is
|
||||
// what makes a saved position land on the title rather than one line in. Rendered
|
||||
// as ordinary prose it reads `第二章 落雨雨来了。`: title and first sentence as one
|
||||
// paragraph, which is what a reader reports as "格式乱了".
|
||||
const html = textToChapterHtml(['第二章 落雨', '雨来了。'].join('\n'));
|
||||
expect(html).toContain('<h3>第二章 落雨</h3>');
|
||||
expect(html).toContain('<p>雨来了。</p>');
|
||||
expect(html).not.toContain('落雨雨来了');
|
||||
});
|
||||
|
||||
it('leaves prose that merely looks like a heading where it is', () => {
|
||||
// Only the *first* line is considered, and only when it is short enough to be a
|
||||
// title. A mid-chapter line that mentions a chapter number is a sentence, and
|
||||
// promoting it would restructure the book rather than render it.
|
||||
const html = textToChapterHtml(['雨来了。', '他说第二章他看过。'].join('\n'));
|
||||
expect(html).not.toContain('<h3>');
|
||||
expect(html).toContain('<p>他说第二章他看过。</p>');
|
||||
});
|
||||
|
||||
it('uses textContent for the node form, so the DOM is the escaper', () => {
|
||||
const element = paragraphElement({ text: '<b>不是标签</b>', leading: 0 });
|
||||
expect(element.tagName).toBe('P');
|
||||
@@ -94,6 +115,37 @@ describe('the markup form', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('splitChapterHeading', () => {
|
||||
it('splits the heading off the body and reports both', () => {
|
||||
// The pair, not just the heading: the body is what the paragraph split then
|
||||
// works on, and returning only the heading would leave the caller to find the
|
||||
// boundary again — a second copy of the rule, which is how the server's copy and
|
||||
// the client's came to disagree in the first place.
|
||||
expect(splitChapterHeading(['第一章 起点', '正文。'].join('\n'))).toEqual({
|
||||
heading: '第一章 起点',
|
||||
body: '正文。',
|
||||
});
|
||||
});
|
||||
|
||||
it('skips leading blank lines before looking for the heading', () => {
|
||||
expect(splitChapterHeading('\n\n第二章 落雨\n正文。').heading).toBe('第二章 落雨');
|
||||
});
|
||||
|
||||
it('reports no heading for a body that has none', () => {
|
||||
expect(splitChapterHeading('正文只有这一段。')).toEqual({
|
||||
heading: '',
|
||||
body: '正文只有这一段。',
|
||||
});
|
||||
});
|
||||
|
||||
it('does not promote a long line that merely starts with a chapter number', () => {
|
||||
// The length cap is what keeps a paragraph that *starts* with `第三章` from being
|
||||
// read as a title, which would move the real body's first line into a heading.
|
||||
const prose = `第三章他记得很清楚,${'那天的雨下得很大,'.repeat(8)}`;
|
||||
expect(splitChapterHeading(prose).heading).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Streaming across chunk boundaries.
|
||||
*
|
||||
|
||||
+15
-3
@@ -231,20 +231,32 @@ describe('plain-text detection', () => {
|
||||
const prose = new ReaderView({ container, doc: textDoc('epub', '<p>他说 div class="txt-body" 是纯文本用的</p>') });
|
||||
expect(prose.isPlainText()).toBe(false);
|
||||
|
||||
// Neither is one with no markup at all.
|
||||
// Neither is one with no markup at all — *unless* the section declares itself.
|
||||
// And that declaration is the case the marker cannot cover: the server now sends
|
||||
// a TXT chapter as bare characters with no wrapper at all, so a chapter that is
|
||||
// nothing but plain text carries nothing to sniff. Sniffing for the marker
|
||||
// answered "not plain text" and the reader drew a novel with no paragraphs, no
|
||||
// indent and no stylesheet — a defect that reads as a typography preference
|
||||
// rather than a bug.
|
||||
const bare = new ReaderView({ container, doc: textDoc('epub', '<p>正文</p>') });
|
||||
expect(bare.isPlainText()).toBe(false);
|
||||
|
||||
const declaredBare = new ReaderView({
|
||||
container,
|
||||
doc: textDoc('epub', '第一章 惊蛰\n\n第 1 段。', { plainText: true }),
|
||||
});
|
||||
expect(declaredBare.isPlainText()).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
/** A one-section document, for the detection test. */
|
||||
function textDoc(format: string, html: string) {
|
||||
function textDoc(format: string, html: string, section: Record<string, unknown> = {}) {
|
||||
return {
|
||||
format,
|
||||
layout: 'reflowable',
|
||||
render: 'reflowable',
|
||||
direction: 'ltr',
|
||||
sections: [{ id: 'c0', label: '一', html, depth: 0 }],
|
||||
sections: [{ id: 'c0', label: '一', html, depth: 0, ...section }],
|
||||
toc: [],
|
||||
styles: [],
|
||||
resources: new Map(),
|
||||
|
||||
+59
-5
@@ -293,15 +293,69 @@ describe('SyncEngine', () => {
|
||||
});
|
||||
|
||||
await engine.syncNow();
|
||||
expect(seen[0]).toBe('POST /api/v1/sync');
|
||||
// The pull reuses the cursor the push just returned. That is intentional: a
|
||||
// pull from 0 would re-download the entire server state after every page
|
||||
// turn, which is exactly what the incremental cursor exists to avoid.
|
||||
expect(seen[1]).toContain('since=5000');
|
||||
// One request, not two.
|
||||
//
|
||||
// `POST /api/v1/sync` already answers with the merged state — the server's own
|
||||
// comment says so — and the engine used to throw that answer away and then
|
||||
// issue a second request for it. Every page turn therefore cost a POST *and* a
|
||||
// GET carrying the same body, which is the doubling a reader sees in the
|
||||
// network panel as "sync 接口在不停重复". Asserting the *length* is what makes
|
||||
// the fix visible: a test that only checked the cursor would pass either way.
|
||||
expect(seen).toEqual(['POST /api/v1/sync']);
|
||||
expect(isOutboxEmpty(offline.current)).toBe(true);
|
||||
// The cursor is still adopted from the push's answer, so the next real pull is
|
||||
// incremental rather than a re-download of the entire server state.
|
||||
expect(offline.current.serverTime).toBe(5000);
|
||||
});
|
||||
|
||||
it('pulls with the incremental cursor when a poll has nothing to push', async () => {
|
||||
// The other half of the same contract: with an empty outbox the push has no
|
||||
// answer to reuse, so the pull is the real work and it must still happen — and
|
||||
// still be incremental.
|
||||
const { api, transport, offline, engine } = build();
|
||||
transport.json(SESSION);
|
||||
await api.login('me', 'password12');
|
||||
await offline.load();
|
||||
|
||||
const seen: string[] = [];
|
||||
transport.respondWith((request) => {
|
||||
seen.push(`${request.method} ${request.url}`);
|
||||
return { status: 200, headers: {}, json: { serverTime: 9000, progress: [], notes: [] } };
|
||||
});
|
||||
|
||||
await engine.syncNow();
|
||||
expect(seen).toEqual(['GET /api/v1/sync?since=0']);
|
||||
expect(offline.current.serverTime).toBe(9000);
|
||||
|
||||
await engine.syncNow();
|
||||
expect(seen[1]).toBe('GET /api/v1/sync?since=9000');
|
||||
});
|
||||
|
||||
it('does not notify its listeners twice for one unchanged state', async () => {
|
||||
// The reported symptom: "sync 接口会不停的重复渲染". The reader screen repaints
|
||||
// its whole chrome tree on every notification, and a cycle announced itself
|
||||
// three times (syncing, idle, and a redundant explicit emit) whether or not
|
||||
// anything had changed — with a 30s poll keeping the cycle going while the
|
||||
// reader did nothing at all.
|
||||
const { api, transport, offline, engine } = build();
|
||||
transport.json(SESSION);
|
||||
await api.login('me', 'password12');
|
||||
await offline.load();
|
||||
transport.json({ serverTime: 10, progress: [], notes: [] });
|
||||
|
||||
const states: string[] = [];
|
||||
engine.onStatus((status) => states.push(`${status.state}:${status.message}`));
|
||||
|
||||
await engine.syncNow();
|
||||
expect(states).toEqual(['syncing:', 'idle:']);
|
||||
|
||||
// A second cycle that ends in the same state as the first adds nothing new, so
|
||||
// it must add nothing to the stream either.
|
||||
const before = states.length;
|
||||
await engine.syncNow();
|
||||
expect(states.slice(before)).toEqual(['syncing:', 'idle:']);
|
||||
});
|
||||
|
||||
it('folds a burst of scheduled pushes into one round trip', async () => {
|
||||
// The reported symptom: "sync 接口调用太过频繁". Every page turn scheduled its
|
||||
// own round trip, so a reader flipping through a book produced a POST and a GET
|
||||
|
||||
@@ -24,6 +24,21 @@ function item(index: number): ContentItem {
|
||||
};
|
||||
}
|
||||
|
||||
/** A plain-text chapter, as the server's manifest declares one. */
|
||||
function textItem(index: number): ContentItem {
|
||||
return {
|
||||
id: `c${index}`,
|
||||
seq: index,
|
||||
title: `第 ${index + 1} 章`,
|
||||
kind: 'chapter',
|
||||
// The declaration the reader typesets from. `format: 'html'` and a `text/plain`
|
||||
// media type are the pair the server actually sends for a TXT.
|
||||
mediaType: 'text/plain; charset=utf-8',
|
||||
href: `chapter:${index}`,
|
||||
format: 'html',
|
||||
};
|
||||
}
|
||||
|
||||
function windowOf(from: number, count: number, total: number): BookContent {
|
||||
return {
|
||||
kind: 'reflowable',
|
||||
@@ -37,6 +52,59 @@ function windowOf(from: number, count: number, total: number): BookContent {
|
||||
}
|
||||
|
||||
describe('staged documents', () => {
|
||||
it('carries the manifest\'s plain-text declaration onto the section', async () => {
|
||||
// The declaration has to reach the *section*, because that is what the reader
|
||||
// asks when it decides whether to typeset. It cannot be re-derived from the body:
|
||||
// the server sends a TXT chapter as bare characters with no wrapper, so a body
|
||||
// that is nothing but plain text carries nothing to recognise it by — and a
|
||||
// reader that answers "not plain text" draws the novel as an unstyled slab.
|
||||
const doc = createStagedDoc({
|
||||
kind: 'reflowable',
|
||||
toc: [],
|
||||
content: {
|
||||
kind: 'reflowable',
|
||||
total: 2,
|
||||
groups: [{ id: 'g', seq: 0, title: '', count: 2, offset: 0 }],
|
||||
items: [textItem(0), item(1)],
|
||||
},
|
||||
loader: { async read() { return { html: '第一章\n\n正文。' }; } },
|
||||
});
|
||||
|
||||
expect(doc.sections[0]?.plainText).toBe(true);
|
||||
// An EPUB chapter in the same window is not swept up by it: the decision is per
|
||||
// section, and an EPUB's own markup must survive untouched.
|
||||
expect(doc.sections[1]?.plainText).toBeUndefined();
|
||||
});
|
||||
|
||||
it('keeps the declaration across a window swap', async () => {
|
||||
// A jump to chapter 900 replaces every section. Losing the flag there would turn
|
||||
// the rest of the book into an EPUB — paragraphs gone, stylesheet gone — and only
|
||||
// after the reader had navigated, which is the hardest kind of defect to report.
|
||||
const doc = createStagedDoc({
|
||||
kind: 'reflowable',
|
||||
toc: [],
|
||||
content: {
|
||||
kind: 'reflowable',
|
||||
total: 4,
|
||||
groups: [{ id: 'g', seq: 0, title: '', count: 2, offset: 0 }],
|
||||
items: [textItem(0), textItem(1)],
|
||||
group: 0,
|
||||
},
|
||||
loader: { async read() { return { html: '第一章\n\n正文。' }; } },
|
||||
});
|
||||
expect(doc.sections[0]?.plainText).toBe(true);
|
||||
|
||||
doc.setWindow({
|
||||
kind: 'reflowable',
|
||||
total: 4,
|
||||
groups: [{ id: 'g', seq: 1, title: '', count: 2, offset: 2 }],
|
||||
items: [textItem(2), textItem(3)],
|
||||
group: 1,
|
||||
}, 2);
|
||||
|
||||
expect(doc.sections[0]?.plainText).toBe(true);
|
||||
});
|
||||
|
||||
it('does not read a section until it is asked for', async () => {
|
||||
const read: string[] = [];
|
||||
const doc = createStagedDoc({
|
||||
|
||||
@@ -156,6 +156,20 @@ export class CDP {
|
||||
return result.result.value;
|
||||
}
|
||||
|
||||
/**
|
||||
* How many requests the stand-in API has answered, by name.
|
||||
*
|
||||
* Read from the page rather than from Node, because that is where the review's own
|
||||
* origin is: the harness server logs every request it sees, and the page is the
|
||||
* only party that can reach it without the tool growing its own HTTP client. The
|
||||
* endpoint returns *counts* rather than the log so the assertion is a comparison of
|
||||
* two numbers instead of a scan of a growing array.
|
||||
*/
|
||||
async requestCounts() {
|
||||
const counts = await this.evaluate(`fetch('/__counts').then((r) => r.json())`);
|
||||
return counts && typeof counts === 'object' ? counts : {};
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs a statement body in the page, with its own `return`.
|
||||
*
|
||||
@@ -267,6 +281,37 @@ export class CDP {
|
||||
await this.send('Input.insertText', { text: value });
|
||||
}
|
||||
|
||||
/**
|
||||
* A horizontal swipe across the middle of the reading area: a page turn.
|
||||
*
|
||||
* Dispatched as *touch* events rather than a click, because that is what the
|
||||
* gesture layer listens for and what a page turn in this app actually is. The
|
||||
* intermediate moves matter: a single `touchStart`/`touchEnd` pair is a tap, and
|
||||
* the gesture layer decides between the two from the distance travelled.
|
||||
*/
|
||||
async swipePage(direction = 'next') {
|
||||
const box = await this.execute(`(() => {
|
||||
const el = document.querySelector('.stage');
|
||||
if (!el) return null;
|
||||
const r = el.getBoundingClientRect();
|
||||
return { x: r.left + r.width / 2, y: r.top + r.height / 2, width: r.width };
|
||||
})()`);
|
||||
if (!box) throw new Error('no .stage to swipe');
|
||||
const travel = box.width * 0.35 * (direction === 'next' ? -1 : 1);
|
||||
const steps = 6;
|
||||
const points = (offset) => [{ x: box.x + offset, y: box.y }];
|
||||
await this.send('Input.dispatchTouchEvent', { type: 'touchStart', touchPoints: points(0) });
|
||||
for (let step = 1; step <= steps; step += 1) {
|
||||
await this.send('Input.dispatchTouchEvent', {
|
||||
type: 'touchMove',
|
||||
touchPoints: points((travel * step) / steps),
|
||||
});
|
||||
await sleep(16);
|
||||
}
|
||||
await this.send('Input.dispatchTouchEvent', { type: 'touchEnd', touchPoints: [] });
|
||||
await sleep(160);
|
||||
}
|
||||
|
||||
/** A tap in the middle of the reading area, which is the chrome toggle. */
|
||||
async tapMiddle() {
|
||||
const box = await this.execute(`(() => {
|
||||
|
||||
@@ -288,6 +288,118 @@ async function audit(cdp, scenes, results) {
|
||||
})()`);
|
||||
check('没有横向溢出', overflow.document <= 1, `document 溢出 ${overflow.document}px`);
|
||||
|
||||
// Back to scroll mode before the traffic window.
|
||||
//
|
||||
// The scenes above end in paged mode, and a paged surface does not scroll — reading
|
||||
// in paged mode turns pages without generating a position per screen. Scroll mode is
|
||||
// where a reader produces a position *continuously*, which is the traffic profile
|
||||
// the report was about, so the window has to be measured there. The sheet is opened
|
||||
// by its own button rather than assumed to be open: the previous step closed it.
|
||||
// The chrome is made visible first: the audit's own collapse check above left it
|
||||
// hidden, and a hidden topbar's settings button is a button that cannot be clicked.
|
||||
const revealChrome = async () => {
|
||||
const hidden = await cdp.evaluate(`document.querySelector('.reader-screen')?.dataset?.chrome === 'hidden'`);
|
||||
if (hidden) {
|
||||
await cdp.tapMiddle();
|
||||
await cdp.sleep(400);
|
||||
}
|
||||
};
|
||||
await revealChrome();
|
||||
// The settings sheet is identified by its own title, not by "a panel is open": the
|
||||
// geometry step above leaves the *contents* sheet up, and a blind click on the
|
||||
// settings button would close it and then wait for a sheet that is already there.
|
||||
const settingsTitle = await cdp.evaluate(
|
||||
`document.querySelector('.panel')?.querySelector('.panel-title, h2, header')?.textContent?.trim() ?? ''`,
|
||||
);
|
||||
if (!settingsTitle.includes('设置')) {
|
||||
if (settingsTitle) {
|
||||
await cdp.click('button[aria-label="关闭"]');
|
||||
await cdp.sleep(300);
|
||||
}
|
||||
await cdp.click('button[aria-label="设置"]');
|
||||
await cdp.waitFor('document.querySelector(".panel") !== null', 10_000);
|
||||
}
|
||||
await cdp.sleep(300);
|
||||
await cdp.clickText('.segmented button', '滚动');
|
||||
await cdp.sleep(300);
|
||||
await cdp.click('button[aria-label="关闭"]');
|
||||
await cdp.sleep(600);
|
||||
|
||||
// The traffic a *reading* session generates.
|
||||
//
|
||||
// The request behind this is the screenshot a reader sent: the network panel, full
|
||||
// of `sync` POSTs each paired with a `sync?since=…` GET, repeating for as long as
|
||||
// they kept reading. Both halves were real defects — the push's answer (which
|
||||
// already carries the merged state) was thrown away and re-fetched, and the 30s
|
||||
// poll was re-armed by every page turn so the pull half silently stopped — and
|
||||
// neither is a *functional* failure, which is why no functional test caught them.
|
||||
//
|
||||
// So the window has to contain actual reading: turning pages is what writes a
|
||||
// position and what used to produce a round trip per turn. A window in which the
|
||||
// reader sits still proves nothing about the thing that was reported.
|
||||
await cdp.sleep(1500);
|
||||
const before = await cdp.requestCounts();
|
||||
// Twelve page turns over six seconds — a reader settling into a book.
|
||||
//
|
||||
// The scroll is dispatched as a real sequence of positions on the reading surface
|
||||
// rather than as a synthetic event: the surface is a scroll container in scroll
|
||||
// mode (that is what "scroll mode" *is*), and its own `scroll` handler is what
|
||||
// measures a position and writes it. Dispatching anything else would be a window in
|
||||
// which no position was ever written, and such a window passes whatever the sync
|
||||
// code does — which is the trap this check has to avoid.
|
||||
// Driven one step at a time from here rather than as one long in-page loop: the
|
||||
// protocol has a 20-second ceiling on a single evaluation, and twelve steps held
|
||||
// long enough for the reader's own write debounce to fire is longer than that.
|
||||
const pageTurns = 8;
|
||||
const scrollMax = await cdp.evaluate(`(() => {
|
||||
const host = document.querySelector('book-content');
|
||||
if (!host) return 0;
|
||||
return Math.max(0, host.scrollHeight - host.clientHeight);
|
||||
})()`);
|
||||
let turned = 0;
|
||||
for (let step = 1; step <= pageTurns && scrollMax > 0; step += 1) {
|
||||
await cdp.evaluate(`(() => {
|
||||
const host = document.querySelector('book-content');
|
||||
host.scrollTop = ${Math.round((scrollMax * step) / pageTurns)};
|
||||
return host.scrollTop;
|
||||
})()`);
|
||||
// Held longer than the reader's own 1.5s write debounce, so every step is a
|
||||
// *committed* position rather than one the next step cancels. Without that the
|
||||
// window contains a single write however many times the surface is scrolled, and
|
||||
// the check has nothing to measure.
|
||||
await cdp.sleep(1800);
|
||||
turned += 1;
|
||||
}
|
||||
// Then let the debounce and the coalescing window drain, so the trailing push a
|
||||
// reader would actually cause is *in* the measurement rather than after it.
|
||||
await cdp.sleep(6000);
|
||||
const after = await cdp.requestCounts();
|
||||
const delta = (name) => (after[name] ?? 0) - (before[name] ?? 0);
|
||||
// Every write costs **one** request, not two.
|
||||
//
|
||||
// This is the doubling the report showed: a `POST /api/v1/sync` immediately
|
||||
// followed by a `GET /api/v1/sync?since=…` carrying the same body. The push already
|
||||
// answers with the merged state, so the GET is entirely redundant — and it doubles
|
||||
// the traffic of every page turn for as long as the reader keeps reading.
|
||||
//
|
||||
// The assertion is on GETs specifically rather than on the total: how many *writes*
|
||||
// a session produces is a product decision that belongs to the coalescing window,
|
||||
// but a GET that appears *alongside* a POST is never right, whatever the reader did.
|
||||
// Every step above is held past the reader's own write debounce, so each is a
|
||||
// separate session and each legitimately pushes; none of them may also pull.
|
||||
const syncTotal = delta('sync');
|
||||
const posts = delta('sync-post');
|
||||
const gets = delta('sync-get');
|
||||
check(
|
||||
'翻页不再每次都发两次 sync',
|
||||
turned > 0 && posts > 0 && gets === 0,
|
||||
turned === 0
|
||||
? '阅读面没有可滚动的高度,这一轮没有发生翻页(检查因此无效)'
|
||||
: posts === 0
|
||||
? '这一轮没有发出任何 push,检查无效'
|
||||
: `${turned} 次翻页 + 6 秒静默:${posts} 次 POST / ${gets} 次 GET(共 ${syncTotal} 次 sync 请求)`,
|
||||
);
|
||||
|
||||
return { failures, results };
|
||||
}
|
||||
|
||||
|
||||
@@ -110,15 +110,21 @@ const book = {
|
||||
updatedAt: Date.now(),
|
||||
};
|
||||
|
||||
/** The reader's own chapter markup, in the shape the server produces. */
|
||||
function chapterHtml(index) {
|
||||
const paragraphs = CHAPTER_BODY[index]
|
||||
.split('\n\n')
|
||||
.map((block) => block.trim())
|
||||
.filter(Boolean)
|
||||
.map((block) => `<p>${block.replace(/&/g, '&').replace(/</g, '<')}</p>`)
|
||||
.join('\n');
|
||||
return `<div class="txt-body">\n${paragraphs}\n</div>\n`;
|
||||
/**
|
||||
* A chapter, in the shape the server actually produces: the characters, as text.
|
||||
*
|
||||
* This used to answer with a `txt-body` div full of `<p>`s, which is what the server
|
||||
* rendered before the typesetting moved to the client. Keeping the fixture on the
|
||||
* *new* shape matters more than keeping it on the old one: the review runs against
|
||||
* the production bundle, and a fixture that still pre-rendered paragraphs would let
|
||||
* a regression in the client's own split render perfectly.
|
||||
*
|
||||
* The `\u3000` indents in `LONG` are kept deliberately — they are "段前空格", and
|
||||
* left in they are indent stacked on the reader's own. The screenshot is where that
|
||||
* is visible.
|
||||
*/
|
||||
function chapterText(index) {
|
||||
return `${CHAPTER_BODY[index]}\n`;
|
||||
}
|
||||
|
||||
function manifest() {
|
||||
@@ -143,8 +149,15 @@ function manifest() {
|
||||
seq: index,
|
||||
title: chapter.title,
|
||||
kind: 'chapter',
|
||||
mediaType: 'application/xhtml+xml',
|
||||
// The TXT manifest as the server actually emits it: plain characters, and
|
||||
// `format: 'html'` to say "this is a document to be typeset" rather than
|
||||
// "this is markup". The media type and the format field are both asserted
|
||||
// here because they are what `renditionRef` keys on — a fixture that omitted
|
||||
// them would have the client fetch `chapter:` and render an untitled slab,
|
||||
// which is a review of a path no reader is on.
|
||||
mediaType: 'text/plain; charset=utf-8',
|
||||
href: `chapter:${index}`,
|
||||
format: 'html',
|
||||
})),
|
||||
},
|
||||
};
|
||||
@@ -174,6 +187,28 @@ export function createReviewServer({ port = 5199 } = {}) {
|
||||
if (path === '/__seen') {
|
||||
return json(reply, seen);
|
||||
}
|
||||
// The request log, *counted*, in the names the traffic assertions use.
|
||||
//
|
||||
// Counting at the HTTP boundary is the only place that sees every request the
|
||||
// bundle makes — including the ones a debounce was supposed to fold away — and
|
||||
// it is deliberately blind to *why* a request was made. A review of the traffic
|
||||
// has to be: the defects it is looking for are a request that was redundant, and
|
||||
// no amount of reading the client's intent tells you whether it was sent.
|
||||
if (path === '/__counts') {
|
||||
const counts = { sync: 0, 'sync-post': 0, 'sync-get': 0, assets: 0, toc: 0 };
|
||||
for (const entry of seen) {
|
||||
const [method, target = ''] = entry.split(' ');
|
||||
const bare = target.split('?')[0] ?? '';
|
||||
if (bare === '/api/v1/sync') {
|
||||
counts.sync += 1;
|
||||
if (method === 'POST') counts['sync-post'] += 1;
|
||||
else counts['sync-get'] += 1;
|
||||
}
|
||||
if (/\/assets$/.test(bare)) counts.assets += 1;
|
||||
if (/\/toc$/.test(bare)) counts.toc += 1;
|
||||
}
|
||||
return json(reply, counts);
|
||||
}
|
||||
if (path.startsWith('/api/')) {
|
||||
return api(request, reply, url);
|
||||
}
|
||||
@@ -250,8 +285,10 @@ export function createReviewServer({ port = 5199 } = {}) {
|
||||
if (path === `/api/v1/books/${BOOK_ID}/assets`) {
|
||||
const ref = url.searchParams.get('ref') ?? '';
|
||||
const index = Number.parseInt(ref.replace(/^chapter(-html)?:/, ''), 10);
|
||||
const body = chapterHtml(Number.isFinite(index) ? index : 0);
|
||||
reply.writeHead(200, { 'content-type': 'text/html; charset=utf-8' });
|
||||
const body = chapterText(Number.isFinite(index) ? index : 0);
|
||||
// `text/plain`, not `text/html`: the server stopped rendering markup, and the
|
||||
// content type is part of what the client is being reviewed against.
|
||||
reply.writeHead(200, { 'content-type': 'text/plain; charset=utf-8' });
|
||||
return reply.end(body);
|
||||
}
|
||||
if (path === `/api/v1/books/${BOOK_ID}/progress`) {
|
||||
@@ -285,4 +322,4 @@ export function createReviewServer({ port = 5199 } = {}) {
|
||||
};
|
||||
}
|
||||
|
||||
export { CHAPTERS, BOOK_ID, CHAPTER_BODY, chapterHtml };
|
||||
export { CHAPTERS, BOOK_ID, CHAPTER_BODY, chapterText };
|
||||
Reference in new issue
Block a user