translate, done with 众生都是既然众生 and 佛教徒的人生态度

skills/dharma-translation/SKILL.md

@@ -56,7 +56,7 @@ Four registers observed, useful as style targets:
 7. **Em-dash convention**: AGENTS.md mandates `—` (Unicode em-dash) → `---` (three hyphens) in English djot. When drafting, type `---` for em-dashes, not `—`. The Chinese source often uses `------` (six hyphens) as its em-dash equivalent — translate to `---`, never to `—`. Before declaring done, run a sanity check: `grep -c '—' target.dj` should be 0.
 8. After translation, offer to align against the terms DB for verification
 
-## Diacritics Convention
+## Sanskrit Italicization\n\nSanskrit/foreign loan words must be italicized on **first occurrence** in the body text. Use `*term*` (djot emphasis). This applies to all non-English Buddhist terms:\n\n- Common: bodhisattva, bodhicitta, samsara, karma, nirvana, Sangha, sutra, Dharma\n- Less common: Mahayana, Sravaka, Vinaya, Lamrim, Ksitigarbha, Samantabhadra, Chan, Arhatship, Theravada\n\nDo NOT italicize subsequent occurrences of the same term. Track which terms have been italicized as you process the body. Only italicize in the running body text, not in TOC, headings, or title lines.\n\nPitfall: some terms like \"karma\" and \"Dharma\" are common enough in English Buddhist\npublishing to appear unitalicized. Follow the convention of the target publication;\nwhen in doubt, italicize on first use.\n\n## Diacritics Convention
 
 Follow the terms DB, not academic Sanskrit. See `references/diacritics-convention.md` for the full rule table. Summary:
 

skills/mpi-project-conventions/SKILL.md

@@ -0,0 +1,134 @@
+---
+name: mpi-project-conventions
+description: Use when working in the MPI project (~/documents/mpi) — translation skill management, terms database, djot conventions, and skill relocation workflow.
+---
+
+# MPI Project Conventions
+
+Project directory: `/home/user/documents/mpi/`
+
+## Skill management
+
+Translation-related skills live in `./skills/` (canonical source). Hermes discovers
+them via `skills.external_dirs` in `~/.hermes/config.yaml`:
+
+```yaml
+skills:
+  external_dirs:
+  - /home/user/documents/mpi/skills
+```
+
+Set with: `hermes config set skills.external_dirs '[/home/user/documents/mpi/skills]'`
+
+Pitfall: `hermes config set` stringifies list values. After running it, verify the
+YAML has proper list syntax (`- /path`, not `'[/path]'`). Edit manually if needed.
+
+The old symlink approach (`~/.hermes/skills/dharma-translation` → `./skills/`) is
+deprecated. `skills/install.fish` has been replaced by `skills/readme.dj`.
+
+## Terms database
+
+- **Module (preferred)**: `from search import search` — call directly in `execute_code` scripts.
+  `search("空性", limit=5, loc="...", src="DoT定稿")` → list of `{zh, en, loc, source}` dicts.
+  No subprocess, no text parsing. Import after `sys.path.insert(0, '/home/user/documents/mpi/terms-search')`.
+- CLI: `/home/user/documents/mpi/terms-search/search.py <query> [limit]`
+- Server: `terms-search/server.py` (Flask, port 8910) — use only when module/CLI is insufficient
+- Start: `python3 /home/user/documents/mpi/terms-search/server.py &`
+- Query: `http://localhost:8910/search?q=...`
+
+## Djot conventions
+
+- Comments use `{% ... %}` syntax
+- Emphasis: `*text*` (single asterisks). `**text**` is Markdown, NOT Djot — never use it.
+- Em dashes: `---` (three hyphens in English text). Pandoc converts to proper em dash in docx output.
+- En dashes: `--` (two hyphens). Pandoc converts to proper en dash in docx output.
+- Preserve source formatting level exactly: if the source has no emphasis on a label, the translation must have none. Do not add or remove formatting.
+- TOC in both `source.dj` and `target.dj`: use clean bullet lists (`- *Section*` / `  - N．item`), not `[text](#anchor)` link markup. Those links are pandoc markdown artifacts. Both files should use the same TOC format.
+- Bilingual files: create `bilingual.dj` alongside `source.dj` and `target.dj`. No new 对照.dj files — existing ones in old projects are artifacts, don't delete them. Generate with `fish scripts/gen-bilingual.fish <article-dir>`. Format: see Bilingual file format section below.
+
+### Markdown → Djot conversion (pandoc)
+
+```bash
+pandoc input.md -f markdown -t djot --wrap=none -o output.dj
+```
+
+Pitfall: pandoc strips `{#id}` attributes from headings but leaves behind stray
+`{#...}` lines. Pre-strip heading anchors from the markdown before conversion:
+
+```bash
+sed 's/ {#[^}]*}//g' input.md | pandoc -f markdown -t djot --wrap=none -o output.dj
+```
+
+Follow up by removing any remaining standalone `{#...}` lines from the djot output:
+
+```bash
+sed -i '/^{#.*}$/d' output.dj
+```
+
+Pitfall — combined documents: When the source `.docx.md` contains multiple articles,
+the TOC at the top often covers all articles. After splitting into per-article
+`source.dj` files, verify each TOC only lists headings that belong to that article.
+Remove entries for sibling articles — the combined TOC is a print-document artifact.
+
+### Bilingual file format (bilingual.dj)
+
+Structure: interleave Chinese source and English target paragraph-by-paragraph.
+
+**Preferred workflow**: when the DOCX manuscript has both languages in 1:1
+correspondence (Chinese, blank, English, blank), extract directly from DOCX.
+No PDF needed — the DOCX English IS the target. See
+`references/proofreading-patterns.md` for the extraction script logic.
+
+**Title & subtitle**: adjacent pair (source, target, no blank between), then a single blank line before the next pair.
+
+**TOC**: source TOC block, blank line, target TOC block — NOT interleaved line-by-line.
+
+**Body**: source line, target line (adjacent — NO blank between them), then a single blank line between pairs.
+
+Pitfall: do NOT put a blank between source and target within a body pair.
+
+**Edit suggestions**: after generating bilingual.dj, scan for issues (garbled text,
+numbering mismatches, translator notes, repeated words) and write
+`edit-suggestions.dj`. Follow the original document's section layout — group
+suggestions under chapter headings, not by issue type. Use diff `-/+` notation.
+
+## Translation skills
+
+Skills tracked in this project:
+- `terms-search` — full-text search across the MPI term database
+- `translation-review` — review CN↔EN translations (CSV/XLSX + .dj comparison)
+- `pptx-translate` — translate PowerPoint files
+- `dharma-translation` — translate Buddhist Dharma talks
+- `chinese-text-normalize` — normalize Chinese markdown line breaks
+- `pdf-to-docx-conversion` — convert PDFs to DOCX with layout preservation
+
+See `references/meditation-translation.md` for lighter workflow when translating
+guided meditation / mindfulness exercise content (vs. Dharma talks).
+
+See `references/translation-pitfalls.md` for recurring CN→EN mistranslation patterns
+(关爱→compassion, 生生增上, 因病返贫, 生存层面, etc.) — review this before starting
+any translation review.
+
+See `references/markdown-to-djot.md` for converting `.docx.md` source files to djot,\nincluding splitting combined articles and cleaning pandoc heading anchors.\n\nSee `references/proofreading-patterns.md` for common manuscript-vs-typeset\ndifferences (term substitutions, numbering changes, typesetting artifacts in\npdftotext output) and the bilingual-from-PDF workflow.
+
+## Utility scripts
+
+Project scripts live in `~/documents/mpi/scripts/`. Write them in fish shell for
+CLI wrappers, Python for data processing.
+
+**Naming**: generic reusable scripts get descriptive names (`dj2docx.fish`,
+`proofread-pdf.py`). Article-specific one-off scripts use `<name>-<hash>.<ext>`
+to signal they're not general-purpose. Don't name a single-article script as if
+it were reusable.
+
+**Agent workflow**: when doing repetitive Python processing (text extraction,
+diffing, data transforms), write the logic to a script in `scripts/` and run it
+via `terminal`. Don't regenerate the same Python in `execute_code` across turns.
+This keeps the agent's output concise — the user sees the results, not the code.
+
+- `dj2docx.fish` — convert `target.dj` → `/tmp/<dirname>-英文.docx` via pandoc.
+  Usage: `fish scripts/dj2docx.fish <path-to-target.dj>`
+- `proofread-pdf.py <docx> <pdf>` — compare manuscript DOCX against typeset PDF.
+- `gen-bilingual-docx.py` — generate `bilingual.dj` directly from DOCX manuscript
+  (English target comes from DOCX, not PDF). Article-specific; name with hash.
+- `gen-bilingual.fish <article-dir>` — generate `bilingual.dj` from `source.dj` + `target.dj`.

skills/mpi-project-conventions/references/markdown-to-djot.md

@@ -0,0 +1,48 @@
+# Markdown to Djot Conversion
+
+When source material arrives as `.docx.md` (pandoc-converted from docx), convert to `.dj` for translation workflows.
+
+## Splitting combined articles
+
+If a single markdown file contains multiple articles (common when docx has two talks in one file), split at the article boundary before converting. Use `sed` by line number:
+
+```bash
+sed -n '1,218p' combined.md > a1.md
+sed -n '220,282p' combined.md > a2.md
+```
+
+## Heading anchor cleanup
+
+Pandoc's docx→md conversion adds `{#heading-id}` anchors to every heading:
+
+```markdown
+## 1．安宁疗护 {#1．安宁疗护}
+```
+
+These must be stripped before markdown→djot conversion, otherwise pandoc's djot writer leaves stray `{#...}` lines in the output:
+
+```bash
+sed 's/ {#[^}]*}//g' input.md > clean.md
+```
+
+## Conversion command
+
+```bash
+pandoc clean.md -f markdown -t djot --wrap=none -o output.dj
+```
+
+`--wrap=none` prevents reflow of long paragraphs.
+
+## Post-conversion cleanup
+
+Pandoc may still leave stray `{#...}` lines in djot output. Remove them:
+
+```bash
+sed -i '/^{#.*}$/d' output.dj
+```
+
+## Pandoc artifacts
+
+- Unicode `——` (U+2014 × 2) → `------` in djot (two em dashes, `---` each). This is correct djot syntax.
+- Markdown hard line breaks (trailing `  `) → `\\\n` in djot. Preserves original paragraph structure.
+- Pandoc normalizes heading IDs (strips `、` and other punctuation). Ignore; the stray-line cleanup handles it.

skills/mpi-project-conventions/references/meditation-translation.md

@@ -0,0 +1,43 @@
+# Meditation / Mindfulness Content Translation
+
+When the source is a guided meditation script, exercise guide, posture instruction,
+or breathing practice (rather than a Dharma talk, sutra commentary, or teaching text),
+use a lighter workflow than the full dharma-translation pipeline.
+
+## Register
+
+Default to warm, direct instructional voice (Thầy-adjacent):
+- Second-person address ("you")
+- Concrete images, sensory details
+- Oral rhythm, short sentences
+- Present tense, imperative mood
+
+MB corpus consultation is NOT needed for register — this content type has its own
+well-established English conventions (yoga/meditation instructional voice).
+
+## Terms
+
+Terms DB lookup for Buddhist-mindfulness vocabulary is useful but limited to key terms:
+- 正念 → mindfulness
+- 觉知 → awareness
+- 无我 → depends on context: "non-self" for philosophical/Dharma content; "selflessly" for embodied/movement instruction where the sense is no separate controller imposing on the action
+- 中道 → Middle Way
+- 丹田 → dantian (keep as-is; well-known in meditation/qigong)
+
+Context-sensitive terms:
+- 心 (xīn): in meditation/movement contexts it often means "mind/attention" not emotional "heart." 持心 means holding the mind with focused attention, not holding with emotion.
+- 念 (niàn): mindfulness/attention/recollection — context between these.
+- Buddhist philosophical terms (无我, 空, 缘起) in non-philosophical contexts (movement instruction, body scans) may need practical/concrete translations rather than doctrinal ones.
+
+Skip deep terms alignment unless dense Dharma vocabulary (emptiness, dependent origination,
+Buddha-nature, etc.) appears in the text.
+
+## Comparison files
+
+Still create 对照.dj as usual. See comparison file format in this skill.
+
+## Pitfalls
+
+- **Don't add formatting the source doesn't have**: sub-section labels using `【】` in Chinese should become plain `[label]` in English, not `*[label]*` or `**[label]**`. Match the source's formatting level exactly.
+- **`**text**` is Markdown, not Djot**: Djot emphasis uses single asterisks (`*text*`). Never use double asterisks in `.dj` files.
+- **心 ≠ heart by default**: in meditation/movement contexts, 持心 = holding the mind with attention, not holding with emotion. Translate based on context, not dictionary defaults.

skills/mpi-project-conventions/references/proofreading-patterns.md

@@ -0,0 +1,63 @@
+# Proofreading: Manuscript vs Typeset
+
+## Two workflows
+
+### A. Bilingual from DOCX (standard)
+
+When the DOCX manuscript has both Chinese and English in 1:1 paragraph
+correspondence, generate `bilingual.dj` directly from the DOCX:
+
+1. `pandoc docx → plain text`
+2. Extract Chinese-English pairs from body (Chinese line, blank, English line, blank)
+3. Apply fixes: italicize Sanskrit on first occurrence, fix `N.Letter` → `N. Letter` spacing
+4. Write bilingual.dj
+
+The DOCX English is the authoritative target text. No PDF needed.
+
+### B. Bilingual from PDF (when PDF is the typeset target)
+
+When the PDF English is the typeset "final" version and should be the target:
+
+1. Extract DOCX Chinese paragraphs (source)
+2. Extract PDF body text via `pdftotext -layout`
+3. Clean PDF: remove slug lines, headers, page numbers, join hyphenation breaks
+4. Match DOCX English paragraphs against PDF body to find positions
+5. Segment PDF body at matched positions
+6. Write bilingual.dj with Chinese source + PDF English target
+
+**Pitfalls in PDF extraction:**
+- Consecutive hyphenation breaks (e.g. `thou-` + `sand...al-` + `leviate`) — the join
+  loop must be recursive: after joining pair N, check if result still ends with `-`
+  and join with line N+2
+- Lines with leading whitespace: use `lstrip()` before checking `n[0].islower()`
+- Drop-cap artifacts: `L iving` → `Living`
+- Trailing section numbers: `...viewpoints. 1)` — the ` 1)` is a PDF section marker
+  bleeding into the previous paragraph
+
+### C. Edit suggestions (edit-suggestions.dj)
+
+After generating bilingual.dj, scan for issues and write `edit-suggestions.dj`:
+
+**Format**: follow the original document's section/chapter layout. Group suggestions
+under the chapter headings where the issues occur. Use diff-style `-/+` notation.
+
+**What to flag:**
+- Garbled Chinese text (merged duplicate edits in source DOCX)
+- Repeated words (`the The`)
+- Chapter numbering mismatches (e.g. `九` ↔ `VIII`)
+- Translator notes in headings (`（善鑫翻，妙一审）`)
+- Missing quotes around dialogue/speech
+
+## Common source DOCX issues
+
+- Translator notes in Chinese headings: `（某某翻，某某审）` — delete for publication
+- Merged duplicate edits: cut-paste errors where old+new text appear together
+- `N.Letter` without space: `2.How` → `2. How`
+- `the The` double article
+
+## Sanskrit italicization
+
+On first occurrence in body text, wrap with `*term*`. Track seen terms across
+the full body. Terms: bodhisattva, bodhicitta, samsara, Dharma, karma, nirvana,
+Sangha, sutra, Mahayana, Sravaka, Vinaya, Lamrim, Ksitigarbha, Samantabhadra,
+Chan, Arhatship, Theravada.

skills/mpi-project-conventions/references/translation-pitfalls.md

@@ -0,0 +1,77 @@
+# Translation Pitfalls — MPI Buddhist Texts
+
+Patterns found in CN→EN translation review. Add to this file as new patterns emerge.
+
+## Terminology conflation
+
+### 关爱/关怀 → compassion (WRONG)
+
+Chinese 关爱 and 关怀 mean "care" or "loving care." They are NOT 慈悲 (compassion / karuṇā).
+Conflating them obscures two distinct Buddhist concepts.
+
+Check every occurrence of "compassion" in a translation against the source:
+- If source is 关爱/关怀 → "care"
+- If source is 慈悲 → "compassion" (correct)
+- If source is 悬壶济世 → "compassionate mission" (correct — the healing spirit)
+
+### 生存层面 → making a living (WRONG)
+
+生存层面 = the existential/survival dimension. Not just earning wages.
+→ "survival-level needs" or "the level of basic existence"
+
+## Loss of Dharma meaning
+
+### 生生增上 → continuously elevate our life (INCOMPLETE)
+
+生生 = life after life (multi-life Buddhist perspective). The single-life rendering
+"continuously elevate our life" loses the Dharma meaning entirely.
+→ "continuously elevate our life, life after life"
+
+## False implication
+
+#### 因病返贫 → "back into poverty"
+"返贫" means becoming poor due to illness, not returning to previous poverty. Use "into poverty" or "driven into poverty."
+
+#### Diacritics: use DB form, not academic Sanskrit
+| Wrong | Right | Source |
+|---|---|---|
+| `Mahāsthāmaprāpta` | `Mahasthamaprapta` | 佛教术语 |
+| `Yogācārabhūmi Śāstra` | `Yogacarabhumi-Sastra` | 经论名 |
+| `Avalokiteśvara` | `Guanyin` | 佛教术语 |
+| `pravāraṇā` | `Pavarana` | BAICKZ |
+
+Exception: `Kṣitigarbha` — DoT定稿 uses diacritics, so keep them.
+When in doubt, search the DB and follow the highest-priority source. See dharma-translation skill `references/diacritics-convention.md`.
+
+返贫 = become poor (from a non-poor state) due to medical costs. "Back" implies
+the person was previously poor — not necessarily true. This is about medical bankruptcy.
+→ "into poverty" or "fall into poverty" (no "back")
+
+## DoT定稿 term drift
+
+### 念死 → recollection of death (WRONG per DoT定稿)
+
+DoT定稿 has "Cultivating mindfulness of death" / 佛教术语 has "contemplating the
+impermanence of death". The established term is "mindfulness of death", not
+"recollection of death." → "mindfulness of death" / "death-mindfulness"
+
+### 三级修学 → Three-Level Study Program (WRONG per DoT定稿)
+
+DoT定稿 has "Three-Stage Practice." → "Three-Stage Practice"
+
+### 下士道/中士道/上士道
+
+DoT定稿: "Path for Persons of Small/Medium/Great Capacity" — not "path of the
+initial/middle/great scope."
+
+### 观音菩萨 → Avalokiteśvara (AVOID in MPI translations)
+
+佛教术语 has "Guanshiyin/Guanyin Bodhisattva." Use "Guanyin Bodhisattva."
+
+## Workflow pitfall
+
+### Translating before consulting terms DB
+
+Always search key terms BEFORE translating. The dharma-translation skill says to do
+this, but it's easy to skip. Use the CLI: `/home/user/documents/mpi/terms-search/search.py <query>`.
+Prioritize DoT定稿 > 内部特色词 > 佛教术语 > 经论名.

skills/translation-review/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: translation-review
-description: Review Chinese-English translations for quality issues - terminology, grammar, consistency, formatting. Two workflows: CSV/XLSX batch review (write .dj suggestions) and .dj comparison line-by-line review (surgical patching).
+description: Review Chinese-English translations for quality issues - terminology, grammar, consistency, formatting. Two workflows - CSV/XLSX batch review (write .dj suggestions) and .dj comparison line-by-line review (surgical patching).
 ---
 
 # Translation Review
@@ -22,7 +22,6 @@ Use `read_file` with offsets for complete coverage. Don't sample.
 ### 3. Write a systematic analysis script
 
 Write to `/tmp/script.py`, run with `python3 /tmp/script.py`. No heredocs or `-c`.
-
 The script should:
 - Parse CSV with `csv.DictReader`
 - Apply detection rules per category
@@ -68,7 +67,7 @@ Use `terminal: cat` — `read_file` deduplicates within a session.
 **Sanity checks first** (mechanical, no judgment needed):
 - **Line count**: source and target must match exactly. Mismatch means paragraphs were dropped, merged, or split.
 - **Em-dash convention**: AGENTS.md says English em-dash (`—`) → three hyphens (`---`). The Chinese source often uses `------` (six hyphens) as its em-dash equivalent — convert to `---` in target, not to a Unicode `—`. A find/replace `—` → `---` over the target file catches all instances at once; a typical long file has 30–50.
-- **TOC format**: AGENTS.md says TOC must be a plain bullet list, no link targets. If target still has `[I. Heading](#...)` markdown links, strip them.
+- **TOC format**: AGENTS.md says TOC must be a plain bullet list, no link targets. If target still has `[I. Heading](#...)` markdown links, strip them. Also check source TOC — per MPI conventions, both source and target should use clean bullet format.
 
 **Terms database drift** (systematic):
 - Cross-reference glossary terms against the MPI terms database
@@ -95,7 +94,9 @@ Use `terminal: cat` — `read_file` deduplicates within a session.
 - Redundant English calques: when the target mirrors a Chinese grammar pattern literally, it can read as a typo (e.g. "mind of death-mindfulness" for 念死之心 — should be "mindfulness of death").
 - Clunky idioms: 一念之差 → "a single thought of difference" is unidiomatic. Standard renderings exist (e.g. "a single errant thought", "a moment's carelessness", or rephrase as "a single thought can make all the difference").
 
-**Missing content**: bare headings with no body — flag, don't invent.
+**Missing content**:
+- **Bare headings** with no body — flag, don't invent.
+- **Mid-paragraph truncation** (common in MPI translations): CN paragraph covers 3–5 clauses but EN stops after 1–2 sentences. Detection: compare semantic density, not character count. CN often packs more meaning per character than EN. Signal: CN has quoted speech, poems, multiple examples, or a rhetorical climax that's absent from EN. Flag as "Missing Content" not "Incomplete" — these are usually draft-stage cutoffs, not intentional omissions.
 
 ### 3. Dump findings to `translation-findings.dj`
 
@@ -110,12 +111,62 @@ Finding N — Title (line numbers)
 
 Surgical string replacement. Verify every patch with `cat` — never rely on `read_file` (session dedup).
 
+### 5. Final sweep
+
+Run `python3 scripts/sweep.py <source.dj> <target.dj> [--stale term1,term2] [--new term1,term2]`. This runs all mechanical checks in one call: line parity, heading parity, Unicode em/en-dashes, Markdown bold, Chinese punctuation, TOC link artifacts, unbalanced quotes, and stale/new term assertions. Run even when no content patches were needed — it serves as final validation.
+
 ## Buddhist terminology reference
 
 See `references/buddhist-terminology.md` for Chinese-English term mappings and common pitfalls.
 
+## Workflow C: Typeset proofread (DOCX manuscript vs PDF layout)
+
+Use when the user gives a manuscript DOCX and a typeset PDF and asks to proofread.
+Goal: catch typesetting errors (missing text, typos, wrong special characters, bad line
+breaks), not translation quality.
+
+### 0. Clarify scope FIRST
+
+Before any extraction: ask what they want checked. "Proofread" can mean:
+- Text accuracy (missing/doubled words, typos introduced by typesetter)
+- Special characters (quotes, dashes, ellipses)
+- Formatting (page numbers, headers, TOC layout)
+- All of the above
+
+Do not run extraction pipelines until scope is clear.
+
+### 1. Extract text
+
+- DOCX → plain: `pandoc file.docx -f docx -t plain --wrap=none`
+- PDF → plain: `pdftotext -layout file.pdf` (preserves positional info)
+
+### 2. Clean PDF artifacts
+
+- Strip InDesign slug lines, page headers, page numbers
+- Join hyphenated line breaks (line ending `-` + next line starting lowercase)
+- Fix drop-cap artifacts (e.g. `L iving` → `Living`)
+
+### 3. Compare
+
+- Extract English paragraphs from DOCX (skip Chinese lines, match blank-line pattern)
+- Check each DOCX paragraph exists as substring in PDF body text
+- Flag paragraphs not found; investigate each (may be heading renumbering, not missing)
+
+### Pitfalls specific to this workflow
+
+- **PDF paragraph joining is lossy** — page breaks split paragraphs. Don't expect
+  perfect paragraph matching; check content coverage, not paragraph identity.
+- **Heading numbering differs** — DOCX has `1.`, `(1)`; PDF has `I`, `1)`. Ignore
+  heading-only differences.
+- **InDesign PDFs insert extra spaces** around drop caps and special characters.
+  Normalize multi-space to single space before comparison.
+
 ## Pitfalls
 
+- **Clarify scope before diving into extraction pipelines** — if the user says
+  "proofread this" or "校对这篇文章", ask what specifically they want checked
+  before running pandoc/pdftotext. Getting interrupted mid-pipeline wastes
+  context.
 - **Don't use heredocs or `-c`** — write to `/tmp/script.py` first
 - **Deduplicate aggressively** — group by problem type, not per-row
 - **Buddhist terminology is technical** — don't guess. When uncertain, flag for review
@@ -125,9 +176,13 @@ See `references/buddhist-terminology.md` for Chinese-English term mappings and c
 - **Em-dash drift**: AGENTS.md mandates `—` (Unicode em-dash) → `---` (three hyphens) in English djot. The Chinese source often uses `------` (six hyphens) as its em-dash equivalent; converters or translators may preserve it as a Unicode `—` in the target, which is a convention violation. Run a single find/replace `—` → `---` over the target. Long files typically have 30–50 such instances.
 - **Batch terminology lookups** — when checking many terms against the terms DB, run them in one `execute_code` script that loops over a query list and calls `search.py` via `subprocess.run`. One terminal call per term floods the context with repetitive output.
 - **Clunky idioms aren't translation errors, they're review items** — a literal calque of a Chinese idiom can read as a typo to a native English reader. Flag these under "Cleanup needed", not "Real errors", and suggest a standard rendering rather than trying to fix in place without confirmation.
-- **Stale-phrasing sweep before declaring done** — after applying patches, run a single script that asserts the target contains zero of the fixed-but-replaced strings, zero Unicode em/en-dashes, and the expected count of the new phrasings. Missed instances (e.g. "mind of death-mindfulness" fixed on L21–L24 but forgotten on L68) survive regular spot-checks. Use `stale = [...]` and `new = [...]` lists; print `[STILL PRESENT (N)]` and `[OK]` per item. Also assert `line_count == source.line_count` and `heading_count == source.heading_count`.
 
 ## References
 
 - `references/buddhist-terminology.md` — Chinese-English Buddhist term mappings and pitfalls
 - `references/terms-db-alignment.md` — Batch-aligning glossary terms against the MPI terms database
+
+## Scripts
+
+- `scripts/sweep.py` — Mechanical validation sweep for completed reviews
+- `scripts/review_csv.py` — Batch CSV/XLSX translation review

skills/translation-review/references/terms-db-alignment.md

@@ -2,36 +2,40 @@
 
 Batch-align translation glossary entries and body text against the MPI terms database.
 
-## Setup
+## Module API (preferred)
 
-Start the HTTP API server if not running:
+Import directly in `execute_code` scripts — no subprocess, no server, no text parsing:
+
+```python
+import sys
+sys.path.insert(0, '/home/user/documents/mpi/terms-search')
+from search import search
+
+results = search("三级修学", limit=5)
+results = search("空性", loc="心经", src="DoT定稿", limit=5)
+# returns list of {zh, en, loc, source} dicts
 ```
-python3 /home/user/documents/mpi/terms-search/server.py &
-```
-Server listens on port 8910.
 
 ## Batch lookup pattern
 
-Use Python via execute_code to query the API for multiple terms:
-
 ```python
-import urllib.request, json, urllib.parse
+import sys
+sys.path.insert(0, '/home/user/documents/mpi/terms-search')
+from search import search
 
 terms = ["三无漏学", "八步三禅", "闻思修", ...]
+author_sources = {"DoT定稿", "内部特色词", "佛教术语", "经论名"}
 
 for term in terms:
-    q = urllib.parse.quote(term)
-    resp = urllib.request.urlopen(f"http://localhost:8910/search?q={q}&limit=5", timeout=10)
-    data = json.loads(resp.read())
-    # Filter to authoritative sources
-    author_sources = ["DoT定稿", "内部特色词", "佛教术语", "经论名"]
-    relevant = [r for r in data["results"] if r["source"] in author_sources]
-    # Compare against current translation, report mismatches
+    results = search(term, limit=10)
+    relevant = [r for r in results if r["source"] in author_sources]
+    for r in relevant:
+        print(f"{r['zh']} → {r['en']}  [{r['source']}]")
 ```
 
-Or with curl:
-```
-curl -s "http://localhost:8910/search?q=三级修学&limit=5" | python3 -c "import sys,json; ..."
+Or filter to a single authoritative source directly:
+```python
+results = search("三级修学", src="DoT定稿", limit=5)
 ```
 
 ## Priority ranking
@@ -56,6 +60,5 @@ When the same term has entries in multiple source tables, prefer:
 ## Pitfalls
 
 - `replace_all` can create doubled words when the surrounding context already contains the replacement string (e.g., "The Eight Steps" → "The The Eight Steps"). Prefer targeted single-replacement patches.
-- The `search.py` CLI does not support `src:` or `loc:` filters — use the HTTP API.
 - Start patches from the bottom of the file upward to preserve line numbers.
 - Some DB entries are contextual phrases (e.g., "珍惜法缘" → a full sentence), not standalone term translations. Use standalone term entries where available.

skills/translation-review/scripts/sweep.py

@@ -0,0 +1,140 @@
+#!/usr/bin/env python3
+"""Mechanical sweep for .dj translation review — run after patches or as final verification.
+
+Usage: python3 sweep.py <source.dj> <target.dj> [--stale term1,term2] [--new term1,term2]
+
+Checks:
+  1. Non-empty line count parity (source == target)
+  2. Heading count parity
+  3. Zero Unicode em-dash (—) / en-dash (–) in target
+  4. Zero Markdown bold (**) in target (djot uses single *)
+  5. Zero common Chinese punctuation in target
+  6. Zero [text](#anchor) link artifacts in target TOC area (first 15 lines)
+  7. Zero unbalanced double-quotes in target
+  8. --stale: each listed string must appear ZERO times in target
+  9. --new: each listed string must appear at least once in target
+"""
+
+import re
+import sys
+
+
+CN_PUNCT = re.compile(r'[\u3000-\u303f\uff00-\uffef\u201c\u201d\u2018\u2019]')
+
+def read_nonempty(path):
+    with open(path) as f:
+        return [l for l in f.read().rstrip('\n').split('\n') if l.strip()]
+
+
+def main():
+    if len(sys.argv) < 3:
+        print("Usage: sweep.py <source.dj> <target.dj> [--stale a,b,c] [--new x,y,z]")
+        sys.exit(2)
+
+    src_path = sys.argv[1]
+    tgt_path = sys.argv[2]
+
+    stale_terms = []
+    new_terms = []
+    i = 3
+    while i < len(sys.argv):
+        if sys.argv[i] == '--stale' and i + 1 < len(sys.argv):
+            stale_terms = [t.strip() for t in sys.argv[i+1].split(',') if t.strip()]
+            i += 2
+        elif sys.argv[i] == '--new' and i + 1 < len(sys.argv):
+            new_terms = [t.strip() for t in sys.argv[i+1].split(',') if t.strip()]
+            i += 2
+        else:
+            i += 1
+
+    src_lines = read_nonempty(src_path)
+    tgt_lines = read_nonempty(tgt_path)
+    tgt_raw = open(tgt_path).read()
+
+    errors = 0
+
+    # 1. Line count
+    if len(src_lines) != len(tgt_lines):
+        print(f"[FAIL] Line count: src={len(src_lines)} tgt={len(tgt_lines)}")
+        errors += 1
+    else:
+        print(f"[OK] Line count: {len(src_lines)}")
+
+    # 2. Heading count
+    src_h = sum(1 for l in src_lines if l.startswith('## '))
+    tgt_h = sum(1 for l in tgt_lines if l.startswith('## '))
+    if src_h != tgt_h:
+        print(f"[FAIL] Headings: src={src_h} tgt={tgt_h}")
+        errors += 1
+    else:
+        print(f"[OK] Headings: {src_h}")
+
+    # 3. Unicode em/en-dash
+    em = tgt_raw.count('\u2014')
+    en = tgt_raw.count('\u2013')
+    if em or en:
+        print(f"[FAIL] Unicode dashes: em-dash={em} en-dash={en}")
+        errors += 1
+    else:
+        print("[OK] No Unicode em/en-dashes")
+
+    # 4. Markdown bold
+    bold = sum(1 for l in tgt_lines if '**' in l)
+    if bold:
+        print(f"[FAIL] Markdown bold (**): {bold} lines")
+        errors += 1
+    else:
+        print("[OK] No Markdown bold")
+
+    # 5. Chinese punctuation
+    cn = [(i+1, l[:60]) for i, l in enumerate(tgt_lines) if CN_PUNCT.search(l)]
+    if cn:
+        print(f"[FAIL] Chinese/smart punct: {len(cn)} lines")
+        for ln, snippet in cn[:5]:
+            print(f"  L{ln}: {snippet}")
+        errors += 1
+    else:
+        print("[OK] No Chinese punctuation")
+
+    # 6. TOC link artifacts (first 15 lines)
+    toc_links = sum(1 for l in tgt_lines[:15] if re.search(r'\[.*?\]\(#', l))
+    if toc_links:
+        print(f"[FAIL] TOC has [text](#anchor) links: {toc_links}")
+        errors += 1
+    else:
+        print("[OK] TOC clean (no link artifacts)")
+
+    # 7. Unbalanced quotes
+    for i, l in enumerate(tgt_lines):
+        if l.count('"') % 2 != 0:
+            print(f"[FAIL] L{i+1}: Unbalanced quotes: {l[:80]}")
+            errors += 1
+    if errors == sum(1 for l in tgt_lines if l.count('"') % 2 != 0):
+        pass  # errors already counted above
+    elif not any(l.count('"') % 2 != 0 for l in tgt_lines):
+        print("[OK] No unbalanced quotes")
+
+    # 8. Stale terms (must be absent)
+    for term in stale_terms:
+        count = tgt_raw.count(term)
+        if count > 0:
+            print(f"[FAIL] Stale term '{term}' still present: {count}")
+            errors += 1
+        else:
+            print(f"[OK] Stale term '{term}' absent")
+
+    # 9. New terms (must be present)
+    for term in new_terms:
+        count = tgt_raw.count(term)
+        if count == 0:
+            print(f"[FAIL] New term '{term}' not found")
+            errors += 1
+        else:
+            print(f"[OK] New term '{term}' found: {count}")
+
+    print(f"\n{'ALL CLEAN' if errors == 0 else f'{errors} ISSUE(S) FOUND'}")
+    sys.exit(0 if errors == 0 else 1)
+
+
+if __name__ == '__main__':
+    main()