translate, done with 众生都是既然众生 and 佛教徒的人生态度

2026-06-15 17:03:59 +08:00
parent 5c87509dcc
commit c9cbf1c2fe
24 changed files with 2873 additions and 191 deletions
@@ -1,6 +1,6 @@
 ---
 name: translation-review
-description: Review Chinese-English translations for quality issues - terminology, grammar, consistency, formatting. Two workflows: CSV/XLSX batch review (write .dj suggestions) and .dj comparison line-by-line review (surgical patching).
+description: Review Chinese-English translations for quality issues - terminology, grammar, consistency, formatting. Two workflows - CSV/XLSX batch review (write .dj suggestions) and .dj comparison line-by-line review (surgical patching).
 ---

 # Translation Review
@@ -22,7 +22,6 @@ Use `read_file` with offsets for complete coverage. Don't sample.
 ### 3. Write a systematic analysis script

 Write to `/tmp/script.py`, run with `python3 /tmp/script.py`. No heredocs or `-c`.
-
 The script should:
 - Parse CSV with `csv.DictReader`
 - Apply detection rules per category
@@ -68,7 +67,7 @@ Use `terminal: cat` — `read_file` deduplicates within a session.
 **Sanity checks first** (mechanical, no judgment needed):
 - **Line count**: source and target must match exactly. Mismatch means paragraphs were dropped, merged, or split.
 - **Em-dash convention**: AGENTS.md says English em-dash (`—`) → three hyphens (`---`). The Chinese source often uses `------` (six hyphens) as its em-dash equivalent — convert to `---` in target, not to a Unicode `—`. A find/replace `—` → `---` over the target file catches all instances at once; a typical long file has 30–50.
- **TOC format**: AGENTS.md says TOC must be a plain bullet list, no link targets. If target still has `[I. Heading](#...)` markdown links, strip them.
+- **TOC format**: AGENTS.md says TOC must be a plain bullet list, no link targets. If target still has `[I. Heading](#...)` markdown links, strip them. Also check source TOC — per MPI conventions, both source and target should use clean bullet format.

 **Terms database drift** (systematic):
 - Cross-reference glossary terms against the MPI terms database
@@ -95,7 +94,9 @@ Use `terminal: cat` — `read_file` deduplicates within a session.
 - Redundant English calques: when the target mirrors a Chinese grammar pattern literally, it can read as a typo (e.g. "mind of death-mindfulness" for 念死之心 — should be "mindfulness of death").
 - Clunky idioms: 一念之差 → "a single thought of difference" is unidiomatic. Standard renderings exist (e.g. "a single errant thought", "a moment's carelessness", or rephrase as "a single thought can make all the difference").

-**Missing content**: bare headings with no body — flag, don't invent.
+**Missing content**:
+- **Bare headings** with no body — flag, don't invent.
+- **Mid-paragraph truncation** (common in MPI translations): CN paragraph covers 3–5 clauses but EN stops after 1–2 sentences. Detection: compare semantic density, not character count. CN often packs more meaning per character than EN. Signal: CN has quoted speech, poems, multiple examples, or a rhetorical climax that's absent from EN. Flag as "Missing Content" not "Incomplete" — these are usually draft-stage cutoffs, not intentional omissions.

 ### 3. Dump findings to `translation-findings.dj`

@@ -110,12 +111,62 @@ Finding N — Title (line numbers)

 Surgical string replacement. Verify every patch with `cat` — never rely on `read_file` (session dedup).

+### 5. Final sweep
+
+Run `python3 scripts/sweep.py <source.dj> <target.dj> [--stale term1,term2] [--new term1,term2]`. This runs all mechanical checks in one call: line parity, heading parity, Unicode em/en-dashes, Markdown bold, Chinese punctuation, TOC link artifacts, unbalanced quotes, and stale/new term assertions. Run even when no content patches were needed — it serves as final validation.
+
 ## Buddhist terminology reference

 See `references/buddhist-terminology.md` for Chinese-English term mappings and common pitfalls.

+## Workflow C: Typeset proofread (DOCX manuscript vs PDF layout)
+
+Use when the user gives a manuscript DOCX and a typeset PDF and asks to proofread.
+Goal: catch typesetting errors (missing text, typos, wrong special characters, bad line
+breaks), not translation quality.
+
+### 0. Clarify scope FIRST
+
+Before any extraction: ask what they want checked. "Proofread" can mean:
+- Text accuracy (missing/doubled words, typos introduced by typesetter)
+- Special characters (quotes, dashes, ellipses)
+- Formatting (page numbers, headers, TOC layout)
+- All of the above
+
+Do not run extraction pipelines until scope is clear.
+
+### 1. Extract text
+
+- DOCX → plain: `pandoc file.docx -f docx -t plain --wrap=none`
+- PDF → plain: `pdftotext -layout file.pdf` (preserves positional info)
+
+### 2. Clean PDF artifacts
+
+- Strip InDesign slug lines, page headers, page numbers
+- Join hyphenated line breaks (line ending `-` + next line starting lowercase)
+- Fix drop-cap artifacts (e.g. `L iving` → `Living`)
+
+### 3. Compare
+
+- Extract English paragraphs from DOCX (skip Chinese lines, match blank-line pattern)
+- Check each DOCX paragraph exists as substring in PDF body text
+- Flag paragraphs not found; investigate each (may be heading renumbering, not missing)
+
+### Pitfalls specific to this workflow
+
+- **PDF paragraph joining is lossy** — page breaks split paragraphs. Don't expect
+  perfect paragraph matching; check content coverage, not paragraph identity.
+- **Heading numbering differs** — DOCX has `1.`, `(1)`; PDF has `I`, `1)`. Ignore
+  heading-only differences.
+- **InDesign PDFs insert extra spaces** around drop caps and special characters.
+  Normalize multi-space to single space before comparison.
+
 ## Pitfalls

+- **Clarify scope before diving into extraction pipelines** — if the user says
+  "proofread this" or "校对这篇文章", ask what specifically they want checked
+  before running pandoc/pdftotext. Getting interrupted mid-pipeline wastes
+  context.
 - **Don't use heredocs or `-c`** — write to `/tmp/script.py` first
 - **Deduplicate aggressively** — group by problem type, not per-row
 - **Buddhist terminology is technical** — don't guess. When uncertain, flag for review
@@ -125,9 +176,13 @@ See `references/buddhist-terminology.md` for Chinese-English term mappings and c
 - **Em-dash drift**: AGENTS.md mandates `—` (Unicode em-dash) → `---` (three hyphens) in English djot. The Chinese source often uses `------` (six hyphens) as its em-dash equivalent; converters or translators may preserve it as a Unicode `—` in the target, which is a convention violation. Run a single find/replace `—` → `---` over the target. Long files typically have 30–50 such instances.
 - **Batch terminology lookups** — when checking many terms against the terms DB, run them in one `execute_code` script that loops over a query list and calls `search.py` via `subprocess.run`. One terminal call per term floods the context with repetitive output.
 - **Clunky idioms aren't translation errors, they're review items** — a literal calque of a Chinese idiom can read as a typo to a native English reader. Flag these under "Cleanup needed", not "Real errors", and suggest a standard rendering rather than trying to fix in place without confirmation.
- **Stale-phrasing sweep before declaring done** — after applying patches, run a single script that asserts the target contains zero of the fixed-but-replaced strings, zero Unicode em/en-dashes, and the expected count of the new phrasings. Missed instances (e.g. "mind of death-mindfulness" fixed on L21–L24 but forgotten on L68) survive regular spot-checks. Use `stale = [...]` and `new = [...]` lists; print `[STILL PRESENT (N)]` and `[OK]` per item. Also assert `line_count == source.line_count` and `heading_count == source.heading_count`.

 ## References

 - `references/buddhist-terminology.md` — Chinese-English Buddhist term mappings and pitfalls
 - `references/terms-db-alignment.md` — Batch-aligning glossary terms against the MPI terms database
+
+## Scripts
+
+- `scripts/sweep.py` — Mechanical validation sweep for completed reviews
+- `scripts/review_csv.py` — Batch CSV/XLSX translation review
@@ -2,36 +2,40 @@

 Batch-align translation glossary entries and body text against the MPI terms database.

-## Setup
+## Module API (preferred)

-Start the HTTP API server if not running:
+Import directly in `execute_code` scripts — no subprocess, no server, no text parsing:
+
+```python
+import sys
+sys.path.insert(0, '/home/user/documents/mpi/terms-search')
+from search import search
+
+results = search("三级修学", limit=5)
+results = search("空性", loc="心经", src="DoT定稿", limit=5)
+# returns list of {zh, en, loc, source} dicts
 ```
-python3 /home/user/documents/mpi/terms-search/server.py &
-```
-Server listens on port 8910.

 ## Batch lookup pattern

-Use Python via execute_code to query the API for multiple terms:
-
 ```python
-import urllib.request, json, urllib.parse
+import sys
+sys.path.insert(0, '/home/user/documents/mpi/terms-search')
+from search import search

 terms = ["三无漏学", "八步三禅", "闻思修", ...]
+author_sources = {"DoT定稿", "内部特色词", "佛教术语", "经论名"}

 for term in terms:
-    q = urllib.parse.quote(term)
-    resp = urllib.request.urlopen(f"http://localhost:8910/search?q={q}&limit=5", timeout=10)
-    data = json.loads(resp.read())
-    # Filter to authoritative sources
-    author_sources = ["DoT定稿", "内部特色词", "佛教术语", "经论名"]
-    relevant = [r for r in data["results"] if r["source"] in author_sources]
-    # Compare against current translation, report mismatches
+    results = search(term, limit=10)
+    relevant = [r for r in results if r["source"] in author_sources]
+    for r in relevant:
+        print(f"{r['zh']} → {r['en']}  [{r['source']}]")
 ```

-Or with curl:
-```
-curl -s "http://localhost:8910/search?q=三级修学&limit=5" | python3 -c "import sys,json; ..."
+Or filter to a single authoritative source directly:
+```python
+results = search("三级修学", src="DoT定稿", limit=5)
 ```

 ## Priority ranking
@@ -56,6 +60,5 @@ When the same term has entries in multiple source tables, prefer:
 ## Pitfalls

 - `replace_all` can create doubled words when the surrounding context already contains the replacement string (e.g., "The Eight Steps" → "The The Eight Steps"). Prefer targeted single-replacement patches.
- The `search.py` CLI does not support `src:` or `loc:` filters — use the HTTP API.
 - Start patches from the bottom of the file upward to preserve line numbers.
 - Some DB entries are contextual phrases (e.g., "珍惜法缘" → a full sentence), not standalone term translations. Use standalone term entries where available.
@@ -0,0 +1,140 @@
+#!/usr/bin/env python3
+"""Mechanical sweep for .dj translation review — run after patches or as final verification.
+
+Usage: python3 sweep.py <source.dj> <target.dj> [--stale term1,term2] [--new term1,term2]
+
+Checks:
+  1. Non-empty line count parity (source == target)
+  2. Heading count parity
+  3. Zero Unicode em-dash (—) / en-dash (–) in target
+  4. Zero Markdown bold (**) in target (djot uses single *)
+  5. Zero common Chinese punctuation in target
+  6. Zero [text](#anchor) link artifacts in target TOC area (first 15 lines)
+  7. Zero unbalanced double-quotes in target
+  8. --stale: each listed string must appear ZERO times in target
+  9. --new: each listed string must appear at least once in target
+"""
+
+import re
+import sys
+
+
+CN_PUNCT = re.compile(r'[\u3000-\u303f\uff00-\uffef\u201c\u201d\u2018\u2019]')
+
+def read_nonempty(path):
+    with open(path) as f:
+        return [l for l in f.read().rstrip('\n').split('\n') if l.strip()]
+
+
+def main():
+    if len(sys.argv) < 3:
+        print("Usage: sweep.py <source.dj> <target.dj> [--stale a,b,c] [--new x,y,z]")
+        sys.exit(2)
+
+    src_path = sys.argv[1]
+    tgt_path = sys.argv[2]
+
+    stale_terms = []
+    new_terms = []
+    i = 3
+    while i < len(sys.argv):
+        if sys.argv[i] == '--stale' and i + 1 < len(sys.argv):
+            stale_terms = [t.strip() for t in sys.argv[i+1].split(',') if t.strip()]
+            i += 2
+        elif sys.argv[i] == '--new' and i + 1 < len(sys.argv):
+            new_terms = [t.strip() for t in sys.argv[i+1].split(',') if t.strip()]
+            i += 2
+        else:
+            i += 1
+
+    src_lines = read_nonempty(src_path)
+    tgt_lines = read_nonempty(tgt_path)
+    tgt_raw = open(tgt_path).read()
+
+    errors = 0
+
+    # 1. Line count
+    if len(src_lines) != len(tgt_lines):
+        print(f"[FAIL] Line count: src={len(src_lines)} tgt={len(tgt_lines)}")
+        errors += 1
+    else:
+        print(f"[OK] Line count: {len(src_lines)}")
+
+    # 2. Heading count
+    src_h = sum(1 for l in src_lines if l.startswith('## '))
+    tgt_h = sum(1 for l in tgt_lines if l.startswith('## '))
+    if src_h != tgt_h:
+        print(f"[FAIL] Headings: src={src_h} tgt={tgt_h}")
+        errors += 1
+    else:
+        print(f"[OK] Headings: {src_h}")
+
+    # 3. Unicode em/en-dash
+    em = tgt_raw.count('\u2014')
+    en = tgt_raw.count('\u2013')
+    if em or en:
+        print(f"[FAIL] Unicode dashes: em-dash={em} en-dash={en}")
+        errors += 1
+    else:
+        print("[OK] No Unicode em/en-dashes")
+
+    # 4. Markdown bold
+    bold = sum(1 for l in tgt_lines if '**' in l)
+    if bold:
+        print(f"[FAIL] Markdown bold (**): {bold} lines")
+        errors += 1
+    else:
+        print("[OK] No Markdown bold")
+
+    # 5. Chinese punctuation
+    cn = [(i+1, l[:60]) for i, l in enumerate(tgt_lines) if CN_PUNCT.search(l)]
+    if cn:
+        print(f"[FAIL] Chinese/smart punct: {len(cn)} lines")
+        for ln, snippet in cn[:5]:
+            print(f"  L{ln}: {snippet}")
+        errors += 1
+    else:
+        print("[OK] No Chinese punctuation")
+
+    # 6. TOC link artifacts (first 15 lines)
+    toc_links = sum(1 for l in tgt_lines[:15] if re.search(r'\[.*?\]\(#', l))
+    if toc_links:
+        print(f"[FAIL] TOC has [text](#anchor) links: {toc_links}")
+        errors += 1
+    else:
+        print("[OK] TOC clean (no link artifacts)")
+
+    # 7. Unbalanced quotes
+    for i, l in enumerate(tgt_lines):
+        if l.count('"') % 2 != 0:
+            print(f"[FAIL] L{i+1}: Unbalanced quotes: {l[:80]}")
+            errors += 1
+    if errors == sum(1 for l in tgt_lines if l.count('"') % 2 != 0):
+        pass  # errors already counted above
+    elif not any(l.count('"') % 2 != 0 for l in tgt_lines):
+        print("[OK] No unbalanced quotes")
+
+    # 8. Stale terms (must be absent)
+    for term in stale_terms:
+        count = tgt_raw.count(term)
+        if count > 0:
+            print(f"[FAIL] Stale term '{term}' still present: {count}")
+            errors += 1
+        else:
+            print(f"[OK] Stale term '{term}' absent")
+
+    # 9. New terms (must be present)
+    for term in new_terms:
+        count = tgt_raw.count(term)
+        if count == 0:
+            print(f"[FAIL] New term '{term}' not found")
+            errors += 1
+        else:
+            print(f"[OK] New term '{term}' found: {count}")
+
+    print(f"\n{'ALL CLEAN' if errors == 0 else f'{errors} ISSUE(S) FOUND'}")
+    sys.exit(0 if errors == 0 else 1)
+
+
+if __name__ == '__main__':
+    main()