iacore
6144d63b68
check-translation.py: 11-check deterministic gate (line/paragraph/heading parity, emphasis preservation, CJK and Chinese-punctuation leakage, bold leakage, 万/亿-aware digit fidelity, terminology vs term map, bilingual freshness). Built and calibrated on the 9-book review batch (2026-08-07). AGENTS.md: add Workflow C (batch translate/review with herdr) — pane setup, the three canonical prompts (translate/review/apply), and gotchas learned on the 9-book run. readme.md: document the gate in common tasks and point to Workflow C.
276 lines
16 KiB
Markdown
276 lines
16 KiB
Markdown
# MPI Project Conventions
|
||
|
||
## Agent Instructions
|
||
|
||
You are working on the Mindful Peace International Chinese-English Buddhist/Dharma translation project.
|
||
- Before translating, load the `mpi-translation` and `mpi-terms-search` skills.
|
||
- Before reviewing, load `mpi-translation-review` (self mode for your own translations, other mode for peer review).
|
||
- The agent IS the model: do not call external translation APIs.
|
||
- The workflow is not as rigid as the state machine below. The user may ask you to deviate from it. Be flexible when asked.
|
||
|
||
## Skills
|
||
|
||
Skills in `toolkit/skills/`.
|
||
|
||
Available: `mpi-translation`, `mpi-terms-search`, `mpi-translation-review`, `mpi-chinese-text-normalize`, `mpi-pptx-translate`, `mpi-pdf-to-docx-conversion`.
|
||
|
||
## Terms Database
|
||
|
||
See `mpi-terms-search` skill. Quick reference:
|
||
- CLI: `toolkit/terms-database/search.py <query> [limit]`
|
||
- Module: `from search import search; search("空性", limit=5, src="DoT定稿")`
|
||
- Priority: DoT定稿 > 内部特色词 > 佛教术语 > 经论名
|
||
|
||
## Directory Structure
|
||
|
||
```
|
||
translate-files/<topic>/<article>/
|
||
source.dj — Chinese original
|
||
target.dj — English translation (line count matches source)
|
||
bilingual.dj — interleaved (source line, blank, target line, blank)
|
||
Generated by `../../toolkit/scripts/gen-bilingual.py source.dj target.dj`;
|
||
do not edit or commit.
|
||
```
|
||
|
||
Put `.docx` output in `/tmp/`. Don't commit binaries.
|
||
Generated files (`bilingual.dj`) are not committed either.
|
||
|
||
---
|
||
|
||
## Translation State Machine
|
||
|
||
Non-deterministic LLM work happens at the states; transitions are fixed.
|
||
|
||
| Current state | Event / condition | Next state | Notes |
|
||
|---|---|---|---|
|
||
| `*start*` | source loaded | `idle` | Begin from a new source. |
|
||
| `*start*` | bilingual loaded | `idle` | Begin from an existing review file. |
|
||
| `idle` | `SOURCE_LOADED` | `translating` | |
|
||
| `idle` | `BILINGUAL_LOADED` | `other_reviewing` | |
|
||
| `translating` | `TRANSLATION_DRAFTED` | `bilingual_ready` | |
|
||
| `bilingual_ready` | `BILINGUAL_GENERATED` | `self_reviewing` | |
|
||
| `self_reviewing` | `SELF_REJECTED` | `translating` | |
|
||
| `self_reviewing` | `SELF_APPROVED` and peer review required | `other_reviewing` | `peer_review_required` flag decides the branch. |
|
||
| `self_reviewing` | `SELF_APPROVED` and no peer review | `approved` | |
|
||
| `other_reviewing` | `PEER_REJECTED` | `translating` | |
|
||
| `other_reviewing` | `PEER_APPROVED` | `approved` | |
|
||
| `approved` | `TYPESET_REQUESTED` | `typesetting` | Optional. |
|
||
| `approved` | `COMPLETE` | `done` | |
|
||
| `typesetting` | `TYPESET_COMPLETE` | `done` | |
|
||
|
||
States:
|
||
|
||
: `idle` — Waiting for source or an existing bilingual file.
|
||
: `translating` — Draft `target.dj`.
|
||
: `bilingual_ready` — `bilingual.dj` generated from `source.dj` + `target.dj`.
|
||
: `self_reviewing` — Self-review with `mpi-translation-review` (self mode); edit `target.dj`.
|
||
: `other_reviewing` — Peer review with `mpi-translation-review` (other mode); write `review-comments.dj`.
|
||
: `approved` — Translation accepted; may typeset or finish.
|
||
: `typesetting` — Produce PDF/DOCX.
|
||
: `done` — Complete.
|
||
|
||
## Workflow A: Translation（翻译）
|
||
|
||
Translate Chinese source into English. The agent IS the model — no external APIs. This workflow covers the state machine path `idle` → `translating` → `bilingual_ready` → `self_reviewing`.
|
||
|
||
### Source context
|
||
|
||
Before translating, the agent must understand the source's format and delivery context. If the source is a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, a written article, or any other genre, that register shapes the translation. If this context is not clear from the file path or source content, ask the user before proceeding.
|
||
|
||
### Input
|
||
|
||
Source text in `.dj` or `.docx` (Chinese only).
|
||
|
||
### Deliverables
|
||
|
||
- `source.dj` — extracted/cleaned Chinese
|
||
- `target.dj` — English translation, line count matches source
|
||
- `bilingual.dj` — interleaved (source line, target line adjacent, blank between pairs).
|
||
Generated by `../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||
Do not create or edit by hand; do not commit.
|
||
- `edit-suggestions.dj` — terminology/consistency issues flagged for review
|
||
|
||
### Rules
|
||
|
||
1. Load `mpi-translation` and `mpi-terms-search` skills before starting.
|
||
2. Search terms DB for key Buddhist terms.
|
||
3. TOC: plain bullet lists, no link targets, no page numbers.
|
||
4. Djot formatting:
|
||
- Emphasis: `*text*` (single asterisks). Never `**` (Markdown bold).
|
||
- Comments: `{% ... %}`
|
||
5. Preserve source formatting — don't add/remove emphasis.
|
||
6. Translate in-response — never call external translation APIs.
|
||
|
||
### Review
|
||
|
||
After translating, load `mpi-translation-review` (self mode) to check:
|
||
- Full detection rules (three passes + R1–R14 editorial polish)
|
||
- Terminology consistency against terms DB
|
||
- Grammar, fluency, calques
|
||
- Missing content (mid-paragraph truncation)
|
||
- Inconsistency (same term translated differently)
|
||
|
||
---
|
||
|
||
## Workflow B: Proofread / Review（校对/审阅）
|
||
|
||
Both self-mode and other-mode review use the unified `mpi-translation-review` skill.
|
||
The **same detection rules** apply to both. The only difference: self mode edits
|
||
`target.dj` directly; other mode writes `review-comments.dj` with collaborative tone.
|
||
|
||
A third mode — **Direct Edit Review** — applies when the user explicitly says to
|
||
edit the `.dj` files directly and skip any comments file.
|
||
|
||
### B1: Self-Review（自审）
|
||
|
||
You translated it. You own the English. Load `mpi-translation-review` skill (self mode).
|
||
|
||
1. Read `source.dj` + `target.dj` fully.
|
||
2. Apply all detection rules (three passes + R1–R14 editorial polish).
|
||
3. Edit `target.dj` directly with `patch` (mode='replace').
|
||
4. Record non-obvious choices in `translation-findings.dj` if needed.
|
||
5. Regenerate `bilingual.dj` with `../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||
6. Verify line counts: `source.dj` and `target.dj` must match.
|
||
|
||
### B2: Other-Review（审他稿）
|
||
|
||
Someone else translated it (volunteer, etc.). Load `mpi-translation-review` skill (other mode).
|
||
|
||
1. Read `source.dj` + `target.dj` fully.
|
||
2. Apply all detection rules (three passes + R1–R14 editorial polish).
|
||
3. Do NOT edit `target.dj` — write `review-comments.dj` instead.
|
||
4. Follow deliberation protocol: 随喜 first, questions not commands.
|
||
5. Address translator by name.
|
||
6. Ask the user whether to apply the findings. If yes, switch to Direct Edit Mode.
|
||
|
||
### B3: Direct Edit Review（直接修改稿）
|
||
|
||
The user wants fixes applied directly, no separate comments file. This can follow
|
||
a translation-review pass, or be a standalone polish pass.
|
||
|
||
1. Read full `target.dj` + `source.dj` in one pass.
|
||
2. Collect every issue using the full detection rules.
|
||
3. Batch all fixes into one set of exact-string replacements. Apply with `patch`.
|
||
4. Regenerate `bilingual.dj` with `../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||
5. Verify line counts match.
|
||
|
||
Avoid iterative "find a few more, edit again" loops. If the user asks "anything
|
||
else?" after a direct-edit pass, do one more full systematic read and batch again.
|
||
|
||
---
|
||
|
||
## Workflow C: Batch Translate / Review with Herdr（批量翻译/审阅）
|
||
|
||
Run one book/article per omp session in its own herdr pane. Proven on the
|
||
9-book batch (2026-08-07): 9 review panes (`omp --model slow`), each moved to
|
||
its own tab, then one apply pass per pane, all 9 green on the check suite.
|
||
|
||
### Setup
|
||
|
||
1. One herdr workspace; the main omp session in the root pane orchestrates.
|
||
2. Rename each book dir with a shared batch prefix so they sort and zip
|
||
together: `batch0-21【《心经》的人生智慧】`, `batch0-55【…】`, …
|
||
3. Split one pane per book. `herdr pane split --cwd <dir>` does NOT stick
|
||
(panes launch in the workspace root) — pass the absolute book dir to omp's
|
||
own `--cwd` at launch instead.
|
||
|
||
```fish
|
||
# 5 right of the main pane, then 4 below it
|
||
r=$(herdr pane split --current --direction right --no-focus)
|
||
p=$(echo "$r" | jq -r '.result.pane.pane_id')
|
||
# repeat: herdr pane split --pane $prev --direction down --no-focus
|
||
herdr pane rename <pane> review-<book> # label each pane
|
||
```
|
||
|
||
4. Launch the slow model in every pane (background all, then `wait`):
|
||
|
||
```fish
|
||
herdr pane run w7:p2 "omp --model slow --cwd /abs/path/to/book-dir" &
|
||
# ... one line per book
|
||
wait
|
||
```
|
||
|
||
Verify each pane landed in its book dir:
|
||
`herdr pane read <pane> --source recent-unwrapped --lines 8` — the TUI title
|
||
shows the dir.
|
||
|
||
5. Submit the review prompt (B below) to all panes at once. Poll
|
||
`herdr pane get <pane> | jq -r '.result.pane.agent_status'` until every
|
||
pane is `idle`/`done` (allow ~1 h for long books).
|
||
6. Move each pane to its own tab so the batch is watchable while it runs:
|
||
`herdr pane move <pane> --new-tab --label <name> --no-focus`.
|
||
7. After all reviews finish, submit the apply prompt (C below) to every pane
|
||
again, poll to completion, then gate with `check-translation.py` and
|
||
spot-check that fixes actually landed in `target.dj`.
|
||
8. Package: `zip -r <batch>-reviewed.zip batch0-*/` and verify the entry count
|
||
(9 books × 6 files each = 63 entries).
|
||
|
||
### The three prompts
|
||
|
||
**A — Translate a book** (one session per book, Workflow A):
|
||
|
||
> Translate the book <NAME> (file: <NAME>.docx) from Chinese to English for the MPI translation project. Follow Workflow A in ../../toolkit/AGENTS.md: (1) load the mpi-translation and mpi-terms-search skills from ../../toolkit/skills/; (2) extract the Chinese source with ../../toolkit/scripts/docx2dj.fish '<NAME>.docx' into source.dj; (3) translate the ENTIRE book into target.dj (English; line count matches source; you ARE the model — no external translation APIs; look up key Buddhist terms with ../../toolkit/terms-database/search.py); (4) generate bilingual.dj: ../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj; (5) self-review with mpi-translation-review (self mode), edit target.dj, and write edit-suggestions.dj for terminology issues; (6) regenerate bilingual.dj and verify source/target line counts match. Deliverables in this folder: source.dj, target.dj, bilingual.dj, edit-suggestions.dj. Do not commit binaries or bilingual.dj. Report when done.
|
||
|
||
**B — Review a book** (herdr pane, slow model; writes `review-findings.dj` only):
|
||
|
||
> Review the translation in this directory (your cwd is the book dir). Files: source.dj (Chinese source), target.dj (English translation), bilingual.dj (bilingual), edit-suggestions.dj (prior edit suggestions, may be stale). Read source and target fully and review the English translation for: (1) accuracy vs source — mistranslations, omissions, additions, meaning drift; (2) Buddhist terminology — consistent, standard renderings; (3) fluency and register — natural, idiomatic English appropriate to the genre; (4) completeness — every source section covered. Write findings to review-findings.dj in this directory, organized by severity (critical/major/minor), each with location and a concrete fix. Do NOT modify source.dj, target.dj, or bilingual.dj. End your final message with a one-paragraph summary.
|
||
|
||
**C — Apply findings** (same pane, direct-edit mode):
|
||
|
||
> Apply your review findings now. This is direct-edit mode per project convention. 1) Read review-findings.dj and target.dj fully. 2) Apply EVERY actionable finding (all must-fix and considerations) to target.dj with exact-string replacements, batched in one pass. 3) CRITICAL: do not add or remove any line — source.dj and target.dj line counts must remain identical. 4) Regenerate bilingual.dj: <abs path>/gen-bilingual.py source.dj target.dj > bilingual.dj 5) Run <abs path>/check-translation.py . and report which checks pass/fail. Report what you changed and the check result.
|
||
|
||
### Gotchas（踩过的坑）
|
||
|
||
- `herdr pane split --cwd <dir>` doesn't stick — launch omp with `--cwd <absolute path>` instead.
|
||
- Write the poll loop carefully: wait for `agent_status` to reach `idle`/`done` with a deadline. The naive first version inverted the logic and reported "done" instantly.
|
||
- Line-count parity is load-bearing: `gen-bilingual.py` pairs lines by index and the check gate FAILs on drift. Apply fixes with exact-string replacements; never insert or delete lines (a blank-line fix was deliberately skipped in the batch for exactly this reason).
|
||
- `check-translation.py` calibration facts (all learned on the 9-book run):
|
||
- CJK leakage ignores djot anchors/links (`{#...}`, `(...)`) — structural markup legitimately contains Chinese.
|
||
- Only strictly-Chinese punctuation flags (`，。、；：？！《》【】（）`); `—` `“”` `’` `·` are legitimate English.
|
||
- Emphasis = preservation on the same line (source `*…*` must survive in target), not count parity — targets legitimately add italics for titles/Sanskrit.
|
||
- Digit fidelity understands 万/亿 scaling, 多, word and comma forms (180亿 → "18 billion", 1300多万 → "13+ million"), and excludes TOC page numbers (`[N](#...)`), which the convention drops.
|
||
- Use `--allow-cjk 人` for intentional Chinese (e.g. a character whose strokes the text explains) and `--term-map term-map.md` to check terminology fidelity; without a term map that check is skipped.
|
||
- `--model slow` is omp's model-role flag for the slow/reasoning model; confirm the exact flag with `omp --help` if unsure.
|
||
- Reviews are the quality gate: the apply pass is what lands findings in `target.dj` (book 21 alone took 26 exact-string replacements). The gate proves mechanics, not quality.
|
||
|
||
---
|
||
|
||
## Djot
|
||
|
||
- Comments: `{% ... %}`
|
||
- Emphasis: `*text*` (single asterisks)
|
||
- Dashes in English: `---` em, `--` en. Pandoc converts in docx output.
|
||
- Preserve source formatting — don't add/remove emphasis
|
||
|
||
When editing `.dj` files, use `patch` (mode='replace') — not regex-based
|
||
string replacement in `execute_code`. `patch` is safer, surfaces conflicts,
|
||
and produces a diff you can review.
|
||
|
||
## Typst Bilingual Template
|
||
|
||
For producing PDFs from bilingual Chinese-English articles:
|
||
|
||
- Template: `translate-files/lib/mpi-bilingual-template.typ`
|
||
- Example host: `translate-files/从物品整理到心灵整理/mindful-organizing.typ`
|
||
- Design notes: `references/typst-template-design.md`
|
||
- Produce rendered PDF files in: /tmp/
|
||
|
||
## Scripts
|
||
|
||
Utility scripts in `toolkit/scripts/` (fish for CLI wrappers, Python for data processing).
|
||
Agents should write repetitive logic here and run via `terminal` rather than
|
||
regenerating the same Python in execute_code each turn.
|
||
|
||
- `toolkit/scripts/docx2dj.fish <docx>` — pandoc .docx → .dj alongside the original
|
||
- `toolkit/scripts/split-bilingual.fish <combined.dj>` — split into source.dj (CN) + target.dj (EN)
|
||
- `toolkit/scripts/dj2docx.fish <target.dj>` — pandoc .dj → .docx in `/tmp/`
|
||
- `toolkit/scripts/proofread-pdf.py <docx> <pdf>` — word-level diff between manuscript and typeset PDF
|
||
- `toolkit/scripts/gen-bilingual.py <source.dj> <target.dj>` — produce `bilingual.dj` on stdout; run as `gen-bilingual.py source.dj target.dj > bilingual.dj`
|
||
- `toolkit/scripts/check-translation.py <book_dir>` — deterministic translation gate: line/paragraph/heading parity, emphasis preservation, CJK & Chinese-punctuation leakage, digit fidelity (万/亿-aware), terminology vs term map, bilingual freshness. Exit 1 on any FAIL. Run before delivering a translation; keep green as a regression suite.
|
||
- `toolkit/scripts/gen-bilingual-<name>-<hash>.py` — article-specific extraction from DOCX or source/target pairing
|
||
- `toolkit/scripts/compile-typst.fish <typ>` — compile a Typst file to PDF
|
||
|
||
Article-specific scripts (including Typst compile helpers) should be placed in
|
||
the article directory itself, named with a short hash: e.g.
|
||
`translate-files/<article>/compile-typst-<hash>.fish`.
|