Compare commits
10
Commits
eaa5e0bf4b
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
07ebbc90c6 | ||
|
|
6144d63b68 | ||
|
|
da0ee57834 | ||
|
|
ee15159856 | ||
|
|
40abd7abe3 | ||
|
|
ef7deb36d8 | ||
|
|
bc17471635 | ||
|
|
894d769051 | ||
|
|
216a7658ac | ||
|
|
8bf19a8d18 |
@@ -39,41 +39,35 @@ Generated files (`bilingual.dj`) are not committed either.
|
|||||||
|
|
||||||
## Translation State Machine
|
## Translation State Machine
|
||||||
|
|
||||||
All translation work follows this deterministic workflow. Non-deterministic LLM work (drafting, reviewing) happens at the edges; the states and transitions are fixed.
|
Non-deterministic LLM work happens at the states; transitions are fixed.
|
||||||
|
|
||||||
```mermaid
|
| Current state | Event / condition | Next state | Notes |
|
||||||
stateDiagram-v2
|
|---|---|---|---|
|
||||||
[*] --> idle
|
| `*start*` | source loaded | `idle` | Begin from a new source. |
|
||||||
idle --> translating: SOURCE_LOADED
|
| `*start*` | bilingual loaded | `idle` | Begin from an existing review file. |
|
||||||
idle --> other_reviewing: BILINGUAL_LOADED
|
| `idle` | `SOURCE_LOADED` | `translating` | |
|
||||||
translating --> bilingual_ready: TRANSLATION_DRAFTED
|
| `idle` | `BILINGUAL_LOADED` | `other_reviewing` | |
|
||||||
bilingual_ready --> self_reviewing: BILINGUAL_GENERATED
|
| `translating` | `TRANSLATION_DRAFTED` | `bilingual_ready` | |
|
||||||
self_reviewing --> translating: SELF_REJECTED
|
| `bilingual_ready` | `BILINGUAL_GENERATED` | `self_reviewing` | |
|
||||||
self_reviewing --> other_reviewing: SELF_APPROVED [peer review required]
|
| `self_reviewing` | `SELF_REJECTED` | `translating` | |
|
||||||
self_reviewing --> approved: SELF_APPROVED [no peer review]
|
| `self_reviewing` | `SELF_APPROVED` and peer review required | `other_reviewing` | `peer_review_required` flag decides the branch. |
|
||||||
note right of self_reviewing
|
| `self_reviewing` | `SELF_APPROVED` and no peer review | `approved` | |
|
||||||
peer_review_required flag decides the branch
|
| `other_reviewing` | `PEER_REJECTED` | `translating` | |
|
||||||
end note
|
| `other_reviewing` | `PEER_APPROVED` | `approved` | |
|
||||||
other_reviewing --> translating: PEER_REJECTED
|
| `approved` | `TYPESET_REQUESTED` | `typesetting` | Optional. |
|
||||||
other_reviewing --> approved: PEER_APPROVED
|
| `approved` | `COMPLETE` | `done` | |
|
||||||
approved --> typesetting: TYPESET_REQUESTED
|
| `typesetting` | `TYPESET_COMPLETE` | `done` | |
|
||||||
approved --> done: COMPLETE
|
|
||||||
typesetting --> done: TYPESET_COMPLETE
|
|
||||||
done --> [*]
|
|
||||||
```
|
|
||||||
|
|
||||||
States:
|
States:
|
||||||
|
|
||||||
| State | Meaning | Output artifact |
|
: `idle` — Waiting for source or an existing bilingual file.
|
||||||
|---|---|---|
|
: `translating` — Draft `target.dj`.
|
||||||
| `idle` | Waiting for source or an existing bilingual file. | — |
|
: `bilingual_ready` — `bilingual.dj` generated from `source.dj` + `target.dj`.
|
||||||
| `translating` | Agent loads `mpi-translation` + `mpi-terms-search` skills and drafts `target.dj`. | `target.dj` |
|
: `self_reviewing` — Self-review with `mpi-translation-review` (self mode); edit `target.dj`.
|
||||||
| `bilingual_ready` | `bilingual.dj` generated from `source.dj` + `target.dj`. | `bilingual.dj` |
|
: `other_reviewing` — Peer review with `mpi-translation-review` (other mode); write `review-comments.dj`.
|
||||||
| `self_reviewing` | Self-review with unified ruleset (self mode). Edit target.dj. | `target.dj` (edited) |
|
: `approved` — Translation accepted; may typeset or finish.
|
||||||
| `other_reviewing` | Peer review with unified ruleset (other mode). Write review-comments.dj. | `review-comments.dj` |
|
: `typesetting` — Produce PDF/DOCX.
|
||||||
| `approved` | Translation accepted. May typeset or finish. | — |
|
: `done` — Complete.
|
||||||
| `typesetting` | Producing PDF/DOCX from approved bilingual content. | `.pdf` / `.docx` |
|
|
||||||
| `done` | Complete. | — |
|
|
||||||
|
|
||||||
## Workflow A: Translation(翻译)
|
## Workflow A: Translation(翻译)
|
||||||
|
|
||||||
@@ -165,6 +159,82 @@ else?" after a direct-edit pass, do one more full systematic read and batch agai
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Workflow C: Batch Translate / Review with Herdr(批量翻译/审阅)
|
||||||
|
|
||||||
|
Run one book/article per omp session in its own herdr pane. Proven on the
|
||||||
|
9-book batch (2026-08-07): 9 review panes (`omp --model slow`), each moved to
|
||||||
|
its own tab, then one apply pass per pane, all 9 green on the check suite.
|
||||||
|
|
||||||
|
### Setup
|
||||||
|
|
||||||
|
1. One herdr workspace; the main omp session in the root pane orchestrates.
|
||||||
|
2. Rename each book dir with a shared batch prefix so they sort and zip
|
||||||
|
together: `batch0-21【《心经》的人生智慧】`, `batch0-55【…】`, …
|
||||||
|
3. Split one pane per book. `herdr pane split --cwd <dir>` does NOT stick
|
||||||
|
(panes launch in the workspace root) — pass the absolute book dir to omp's
|
||||||
|
own `--cwd` at launch instead.
|
||||||
|
|
||||||
|
```fish
|
||||||
|
# 5 right of the main pane, then 4 below it
|
||||||
|
r=$(herdr pane split --current --direction right --no-focus)
|
||||||
|
p=$(echo "$r" | jq -r '.result.pane.pane_id')
|
||||||
|
# repeat: herdr pane split --pane $prev --direction down --no-focus
|
||||||
|
herdr pane rename <pane> review-<book> # label each pane
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Launch the slow model in every pane (background all, then `wait`):
|
||||||
|
|
||||||
|
```fish
|
||||||
|
herdr pane run w7:p2 "omp --model slow --cwd /abs/path/to/book-dir" &
|
||||||
|
# ... one line per book
|
||||||
|
wait
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify each pane landed in its book dir:
|
||||||
|
`herdr pane read <pane> --source recent-unwrapped --lines 8` — the TUI title
|
||||||
|
shows the dir.
|
||||||
|
|
||||||
|
5. Submit the review prompt (B below) to all panes at once. Poll
|
||||||
|
`herdr pane get <pane> | jq -r '.result.pane.agent_status'` until every
|
||||||
|
pane is `idle`/`done` (allow ~1 h for long books).
|
||||||
|
6. Move each pane to its own tab so the batch is watchable while it runs:
|
||||||
|
`herdr pane move <pane> --new-tab --label <name> --no-focus`.
|
||||||
|
7. After all reviews finish, submit the apply prompt (C below) to every pane
|
||||||
|
again, poll to completion, then gate with `check-translation.py` and
|
||||||
|
spot-check that fixes actually landed in `target.dj`.
|
||||||
|
8. Package: `zip -r <batch>-reviewed.zip batch0-*/` and verify the entry count
|
||||||
|
(9 books × 6 files each = 63 entries).
|
||||||
|
|
||||||
|
### The three prompts
|
||||||
|
|
||||||
|
**A — Translate a book** (one session per book, Workflow A):
|
||||||
|
|
||||||
|
> Translate the book <NAME> (file: <NAME>.docx) from Chinese to English for the MPI translation project. Follow Workflow A in ../../toolkit/AGENTS.md: (1) load the mpi-translation and mpi-terms-search skills from ../../toolkit/skills/; (2) extract the Chinese source with ../../toolkit/scripts/docx2dj.fish '<NAME>.docx' into source.dj; (3) translate the ENTIRE book into target.dj (English; line count matches source; you ARE the model — no external translation APIs; look up key Buddhist terms with ../../toolkit/terms-database/search.py); (4) generate bilingual.dj: ../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj; (5) self-review with mpi-translation-review (self mode), edit target.dj, and write edit-suggestions.dj for terminology issues; (6) regenerate bilingual.dj and verify source/target line counts match. Deliverables in this folder: source.dj, target.dj, bilingual.dj, edit-suggestions.dj. Do not commit binaries or bilingual.dj. Report when done.
|
||||||
|
|
||||||
|
**B — Review a book** (herdr pane, slow model; writes `review-findings.dj` only):
|
||||||
|
|
||||||
|
> Review the translation in this directory (your cwd is the book dir). Files: source.dj (Chinese source), target.dj (English translation), bilingual.dj (bilingual), edit-suggestions.dj (prior edit suggestions, may be stale). Read source and target fully and review the English translation for: (1) accuracy vs source — mistranslations, omissions, additions, meaning drift; (2) Buddhist terminology — consistent, standard renderings; (3) fluency and register — natural, idiomatic English appropriate to the genre; (4) completeness — every source section covered. Write findings to review-findings.dj in this directory, organized by severity (critical/major/minor), each with location and a concrete fix. Do NOT modify source.dj, target.dj, or bilingual.dj. End your final message with a one-paragraph summary.
|
||||||
|
|
||||||
|
**C — Apply findings** (same pane, direct-edit mode):
|
||||||
|
|
||||||
|
> Apply your review findings now. This is direct-edit mode per project convention. 1) Read review-findings.dj and target.dj fully. 2) Apply EVERY actionable finding (all must-fix and considerations) to target.dj with exact-string replacements, batched in one pass. 3) CRITICAL: do not add or remove any line — source.dj and target.dj line counts must remain identical. 4) Regenerate bilingual.dj: <abs path>/gen-bilingual.py source.dj target.dj > bilingual.dj 5) Run <abs path>/check-translation.py . and report which checks pass/fail. Report what you changed and the check result.
|
||||||
|
|
||||||
|
### Gotchas(踩过的坑)
|
||||||
|
|
||||||
|
- `herdr pane split --cwd <dir>` doesn't stick — launch omp with `--cwd <absolute path>` instead.
|
||||||
|
- Write the poll loop carefully: wait for `agent_status` to reach `idle`/`done` with a deadline. The naive first version inverted the logic and reported "done" instantly.
|
||||||
|
- Line-count parity is load-bearing: `gen-bilingual.py` pairs lines by index and the check gate FAILs on drift. Apply fixes with exact-string replacements; never insert or delete lines (a blank-line fix was deliberately skipped in the batch for exactly this reason).
|
||||||
|
- `check-translation.py` calibration facts (all learned on the 9-book run):
|
||||||
|
- CJK leakage ignores djot anchors/links (`{#...}`, `(...)`) — structural markup legitimately contains Chinese.
|
||||||
|
- Only strictly-Chinese punctuation flags (`,。、;:?!《》【】()`); `—` `“”` `’` `·` are legitimate English.
|
||||||
|
- Emphasis = preservation on the same line (source `*…*` must survive in target), not count parity — targets legitimately add italics for titles/Sanskrit.
|
||||||
|
- Digit fidelity understands 万/亿 scaling, 多, word and comma forms (180亿 → "18 billion", 1300多万 → "13+ million"), and excludes TOC page numbers (`[N](#...)`), which the convention drops.
|
||||||
|
- Use `--allow-cjk 人` for intentional Chinese (e.g. a character whose strokes the text explains) and `--term-map term-map.md` to check terminology fidelity; without a term map that check is skipped.
|
||||||
|
- `--model slow` is omp's model-role flag for the slow/reasoning model; confirm the exact flag with `omp --help` if unsure.
|
||||||
|
- Reviews are the quality gate: the apply pass is what lands findings in `target.dj` (book 21 alone took 26 exact-string replacements). The gate proves mechanics, not quality.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Djot
|
## Djot
|
||||||
|
|
||||||
- Comments: `{% ... %}`
|
- Comments: `{% ... %}`
|
||||||
@@ -196,6 +266,7 @@ regenerating the same Python in execute_code each turn.
|
|||||||
- `toolkit/scripts/dj2docx.fish <target.dj>` — pandoc .dj → .docx in `/tmp/`
|
- `toolkit/scripts/dj2docx.fish <target.dj>` — pandoc .dj → .docx in `/tmp/`
|
||||||
- `toolkit/scripts/proofread-pdf.py <docx> <pdf>` — word-level diff between manuscript and typeset PDF
|
- `toolkit/scripts/proofread-pdf.py <docx> <pdf>` — word-level diff between manuscript and typeset PDF
|
||||||
- `toolkit/scripts/gen-bilingual.py <source.dj> <target.dj>` — produce `bilingual.dj` on stdout; run as `gen-bilingual.py source.dj target.dj > bilingual.dj`
|
- `toolkit/scripts/gen-bilingual.py <source.dj> <target.dj>` — produce `bilingual.dj` on stdout; run as `gen-bilingual.py source.dj target.dj > bilingual.dj`
|
||||||
|
- `toolkit/scripts/check-translation.py <book_dir>` — deterministic translation gate: line/paragraph/heading parity, emphasis preservation, CJK & Chinese-punctuation leakage, digit fidelity (万/亿-aware), terminology vs term map, bilingual freshness. Exit 1 on any FAIL. Run before delivering a translation; keep green as a regression suite.
|
||||||
- `toolkit/scripts/gen-bilingual-<name>-<hash>.py` — article-specific extraction from DOCX or source/target pairing
|
- `toolkit/scripts/gen-bilingual-<name>-<hash>.py` — article-specific extraction from DOCX or source/target pairing
|
||||||
- `toolkit/scripts/compile-typst.fish <typ>` — compile a Typst file to PDF
|
- `toolkit/scripts/compile-typst.fish <typ>` — compile a Typst file to PDF
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# Idea: distill Master Jiqun's way of talk (语气) into a skill
|
||||||
|
|
||||||
|
Date: 2026-07-20
|
||||||
|
Status: idea / not scheduled
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
Reviewer feedback on a translated article: translations must preserve Master
|
||||||
|
Jiqun's 语气 — the speaker's voice and manner — not just the tone (register,
|
||||||
|
formality). A translation can have the right register and still flatten the
|
||||||
|
teacher's voice.
|
||||||
|
|
||||||
|
As a first step, a "Speaker's Voice (语气)" section was added to
|
||||||
|
`skills/mpi-translation/SKILL.md`, and check D3 in
|
||||||
|
`skills/mpi-translation-review/SKILL.md` was expanded to verify 语气 markers.
|
||||||
|
That covers the immediate need, but the knowledge is currently a short bullet
|
||||||
|
list inside a general skill.
|
||||||
|
|
||||||
|
## Idea
|
||||||
|
|
||||||
|
Distill Master Jiqun's way of talk into a dedicated skill (or a reference doc
|
||||||
|
under `skills/mpi-translation/references/`), built from evidence across many
|
||||||
|
of his talks:
|
||||||
|
|
||||||
|
- Collect characteristic passages (source + approved translations) from the
|
||||||
|
terms DB (`loc` fields) and finished articles in `translate-files/`.
|
||||||
|
- Catalog his recurring devices: rhetorical question chains, first-person
|
||||||
|
asides (我经常说 / 我曾在讲座中 / 由此我想到), reasoning connectives
|
||||||
|
(可见 / 所以说 / 事实上), homely analogies (rotting apple, leaking boat,
|
||||||
|
teacup, face mask), gentle admonition with humor, measured unhurried
|
||||||
|
authority, inclusive we/you address.
|
||||||
|
- For each device, give approved English renderings and common failure modes
|
||||||
|
(e.g. question → declaration, "我经常说" → "it is said").
|
||||||
|
- Possibly also cover delivery/register differences between his oral talks,
|
||||||
|
essays, and micro-blog posts.
|
||||||
|
|
||||||
|
## Open questions
|
||||||
|
|
||||||
|
- Standalone skill vs. reference doc under `mpi-translation`? A reference doc
|
||||||
|
is probably enough; a full skill may be overkill.
|
||||||
|
- Who approves the example translations used as evidence?
|
||||||
|
- Should it also cover other MPI teachers, or stay Jiqun-specific?
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
# MPI Translation Toolkit
|
||||||
|
|
||||||
|
This is the public translation toolkit for Mindful Peace International (MPI).
|
||||||
|
It contains agent skills, a Buddhist/Dharma term database, helper scripts, and
|
||||||
|
project conventions. You clone it into a private workspace and work on your
|
||||||
|
translations in sibling directories.
|
||||||
|
|
||||||
|
## Who this is for
|
||||||
|
|
||||||
|
Volunteers who want to translate MPI articles using the Oh My Pi agent
|
||||||
|
workflow. You do not need to know Oh My Pi yet; this guide covers the setup.
|
||||||
|
|
||||||
|
## Workspace layout
|
||||||
|
|
||||||
|
Create a workspace directory and clone this repo into `toolkit/` inside it.
|
||||||
|
Your translations and reference materials live next to `toolkit/`, not inside
|
||||||
|
it. Treat `toolkit/` as read-only unless you are intentionally contributing to
|
||||||
|
the toolkit itself.
|
||||||
|
|
||||||
|
```text
|
||||||
|
mpi-workspace/
|
||||||
|
├── toolkit/ ← this repository (public, read-only for translators)
|
||||||
|
│ ├── skills/
|
||||||
|
│ ├── terms-database/
|
||||||
|
│ ├── scripts/
|
||||||
|
│ └── AGENTS.md
|
||||||
|
├── translate-files/ ← your translation projects
|
||||||
|
└── references/ ← your private reference materials
|
||||||
|
```
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
mkdir ~/mpi-workspace
|
||||||
|
cd ~/mpi-workspace
|
||||||
|
git clone https://codeberg.org/eastwind/translation-toolkit.git toolkit
|
||||||
|
mkdir translate-files references
|
||||||
|
```
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
Install these before you start:
|
||||||
|
|
||||||
|
- **Oh My Pi** — the agent harness you will talk to.
|
||||||
|
- **Python 3** and **uv** — used by the scripts.
|
||||||
|
- **fish** — the scripts in `toolkit/scripts/` are written for fish.
|
||||||
|
- **pandoc** — converts between `.docx` and `.dj` (djot).
|
||||||
|
- **typst** — compiles bilingual layouts to PDF.
|
||||||
|
|
||||||
|
## Load the skills
|
||||||
|
|
||||||
|
Configure your agent to load skills from the `toolkit/skills/` directory. The
|
||||||
|
skills use the standard Agent Skills format. If your agent supports an
|
||||||
|
`external_dirs`-style skill path, add the absolute path:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
skills:
|
||||||
|
external_dirs:
|
||||||
|
- /home/yourname/mpi-workspace/toolkit/skills
|
||||||
|
```
|
||||||
|
|
||||||
|
Replace `/home/yourname/mpi-workspace` with the actual absolute path to your
|
||||||
|
workspace.
|
||||||
|
|
||||||
|
## First walkthrough
|
||||||
|
|
||||||
|
Use the shortest existing article to verify your setup:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cd ~/mpi-workspace/toolkit
|
||||||
|
ls ../translate-files/展示
|
||||||
|
# source.dj target.dj
|
||||||
|
```
|
||||||
|
|
||||||
|
A minimal project has these files:
|
||||||
|
|
||||||
|
```text
|
||||||
|
../translate-files/展示/
|
||||||
|
├── source.dj ← Chinese original, one paragraph per line
|
||||||
|
└── target.dj ← English translation, matching source line count
|
||||||
|
```
|
||||||
|
|
||||||
|
Generate the bilingual file:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
./scripts/gen-bilingual.py \
|
||||||
|
../translate-files/展示/source.dj \
|
||||||
|
../translate-files/展示/target.dj \
|
||||||
|
> ../translate-files/展示/bilingual.dj
|
||||||
|
```
|
||||||
|
|
||||||
|
The output interleaves source, target, and blank lines. `bilingual.dj` is
|
||||||
|
generated; do not edit it by hand or commit it.
|
||||||
|
|
||||||
|
## Common tasks
|
||||||
|
|
||||||
|
| Task | Command |
|
||||||
|
|---|---|
|
||||||
|
| Convert `.docx` to `.dj` | `./scripts/docx2dj.fish input.docx output.dj` |
|
||||||
|
| Convert `target.dj` to `.docx` | `./scripts/dj2docx.fish ../translate-files/展示/target.dj` |
|
||||||
|
| Compile a Typst file to PDF | `./scripts/compile-typst.fish ../translate-files/my-article/my-article.typ` |
|
||||||
|
| Run the translation gate | `./scripts/check-translation.py ../translate-files/my-article/` — 11 deterministic checks; exit 1 on any FAIL |
|
||||||
|
| Search the term database | `./terms-database/search.py 空性` |
|
||||||
|
| Run the term database UI | `./terms-database/server.py` then open <http://127.0.0.1:8910> |
|
||||||
|
|
||||||
|
## Sharing your work
|
||||||
|
|
||||||
|
Keep your workspace private. Do not share translation files through this
|
||||||
|
repository. Share finished translations with the team through WeChat or Google
|
||||||
|
Docs, as arranged by your project coordinator.
|
||||||
|
|
||||||
|
## Learn more
|
||||||
|
|
||||||
|
- `AGENTS.md` — full MPI project conventions, translation workflows, review rules, and Workflow C: the herdr batch workflow for translating or reviewing many books in parallel (one omp pane per book).
|
||||||
|
- `skills/readme.dj` — how the skills are organized.
|
||||||
|
- `references/` — design notes, formatting guides, and other reference materials.
|
||||||
|
|
||||||
|
If you improve the toolkit itself, contributions back to the Codeberg repo are
|
||||||
|
welcome. If you are only translating, leave `toolkit/` unchanged.
|
||||||
Executable
+411
@@ -0,0 +1,411 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Deterministic translation checks for the MPI project (TDD-style gate).
|
||||||
|
|
||||||
|
Checks the mechanical invariants of a Chinese->English djot translation.
|
||||||
|
Semantic quality (fluency, register, tone) is NOT checked here — that is the
|
||||||
|
LLM review layer. This script is the hard gate: it must go green before a
|
||||||
|
translation is delivered, and it stays in toolkit/scripts/ as a regression
|
||||||
|
suite so later edits cannot silently break parity.
|
||||||
|
|
||||||
|
Severity:
|
||||||
|
FAIL — hard invariant broken; gate is red.
|
||||||
|
WARN — possible drift worth a reviewer's eye; does not fail the gate.
|
||||||
|
PASS — clean.
|
||||||
|
|
||||||
|
Checks:
|
||||||
|
1. line-count parity — source.dj and target.dj must have equal lines
|
||||||
|
2. paragraph parity — equal number of blank-separated blocks
|
||||||
|
3. heading parity — equal # of lines starting with each heading level
|
||||||
|
4. emphasis preservation — every *...* in a source line survives on the
|
||||||
|
same-index target line (D5). Target may ADD
|
||||||
|
italics (titles, Sanskrit) — that is fine.
|
||||||
|
5. comment parity — equal {% ... %} blocks (D5)
|
||||||
|
6. CJK leakage — no Chinese characters in target (whitelistable)
|
||||||
|
7. Chinese punctuation — no strictly-Chinese punctuation in target
|
||||||
|
(,。、;:?!《》【】()). Em dash, curly
|
||||||
|
quotes, middot are legal English — not flagged.
|
||||||
|
8. bold leakage — no Markdown ** in target (D5)
|
||||||
|
9. digit fidelity — every arabic number in source appears in target
|
||||||
|
10. terminology — source terms from a term map must appear in target
|
||||||
|
with an allowed English rendering (A3 / terms DB).
|
||||||
|
FAIL: term present in source, none of its
|
||||||
|
renderings found in target. WARN: renderings
|
||||||
|
found but fewer times than the source term.
|
||||||
|
11. bilingual freshness — bilingual.dj, if present, equals a regeneration
|
||||||
|
from source+target
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
check-translation.py <book_dir>
|
||||||
|
Uses <book_dir>/source.dj and <book_dir>/target.dj; auto-detects
|
||||||
|
bilingual.dj and term-map.md in the same directory.
|
||||||
|
check-translation.py <source.dj> <target.dj> [--bilingual FILE]
|
||||||
|
Explicit files.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--term-map FILE Term map (default: <book_dir>/term-map.md if present).
|
||||||
|
Accepted formats:
|
||||||
|
- markdown table rows: | 菩提心 | bodhicitta |
|
||||||
|
- plain lines: CN<TAB>EN or CN|EN1|EN2
|
||||||
|
Multiple Chinese terms separated by "/" or "、" share
|
||||||
|
one English side; English renderings separated by "/"
|
||||||
|
are alternatives, any of which satisfies the check.
|
||||||
|
--allow-cjk LIST Comma-separated CJK strings permitted in target
|
||||||
|
(e.g. quoted book titles like 《心经》).
|
||||||
|
--json Emit machine-readable JSON results.
|
||||||
|
|
||||||
|
Exit code: 0 when no check FAILs (WARNs allowed), 1 otherwise.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
CJK_RE = re.compile(r"[\u3400-\u4dbf\u4e00-\u9fff\uf900-\ufaff]")
|
||||||
|
# Strictly-Chinese punctuation only. Em dash (—), curly quotes (“” ‘’),
|
||||||
|
# middot (·), and ellipsis are legitimate in English prose.
|
||||||
|
CN_PUNCT_RE = re.compile(r"[,。、;:?!《》【】()]")
|
||||||
|
EMPHASIS_RE = re.compile(r"\*[^*\n]+\*")
|
||||||
|
COMMENT_RE = re.compile(r"\{%[\s\S]*?%\}")
|
||||||
|
HEADING_RE = re.compile(r"^(#{1,6})(?:\s|$)")
|
||||||
|
BOLD_RE = re.compile(r"\*\*")
|
||||||
|
DIGIT_RE = re.compile(r"\d+")
|
||||||
|
|
||||||
|
PASS, WARN, FAIL = "PASS", "WARN", "FAIL"
|
||||||
|
|
||||||
|
|
||||||
|
def read_lines(path):
|
||||||
|
return Path(path).read_text(encoding="utf-8").splitlines()
|
||||||
|
|
||||||
|
|
||||||
|
def count_paras(lines):
|
||||||
|
"""Blank-separated blocks; leading/trailing blanks ignored."""
|
||||||
|
count = 0
|
||||||
|
in_block = False
|
||||||
|
for line in lines:
|
||||||
|
if line.strip():
|
||||||
|
if not in_block:
|
||||||
|
count += 1
|
||||||
|
in_block = True
|
||||||
|
else:
|
||||||
|
in_block = False
|
||||||
|
return count
|
||||||
|
|
||||||
|
|
||||||
|
def heading_counts(lines):
|
||||||
|
counts = {i: 0 for i in range(1, 7)}
|
||||||
|
for line in lines:
|
||||||
|
m = HEADING_RE.match(line)
|
||||||
|
if m:
|
||||||
|
counts[len(m.group(1))] += 1
|
||||||
|
return counts
|
||||||
|
|
||||||
|
|
||||||
|
def term_map_from_markdown(text):
|
||||||
|
"""Parse a term map into {chinese_term: [english_renderings]}."""
|
||||||
|
terms = {}
|
||||||
|
for raw in text.splitlines():
|
||||||
|
line = raw.strip()
|
||||||
|
if not line or line.startswith("#"):
|
||||||
|
continue
|
||||||
|
if "\t" in line:
|
||||||
|
cn, _, en = line.partition("\t")
|
||||||
|
terms[cn.strip()] = [r.strip() for r in en.split("/") if r.strip()]
|
||||||
|
continue
|
||||||
|
if "|" not in line:
|
||||||
|
continue
|
||||||
|
cells = [c.strip() for c in line.strip("|").split("|")]
|
||||||
|
if len(cells) < 2:
|
||||||
|
continue
|
||||||
|
cn_cell, en_cell = cells[0], cells[1]
|
||||||
|
if not cn_cell or not en_cell or set(cn_cell) <= {"-", " "}:
|
||||||
|
continue
|
||||||
|
for cn in (c.strip() for c in re.split(r"[/、]", cn_cell) if c.strip()):
|
||||||
|
terms[cn] = [r.strip() for r in en_cell.split("/") if r.strip()]
|
||||||
|
return terms
|
||||||
|
|
||||||
|
|
||||||
|
def check_line_count(src, tgt):
|
||||||
|
ok = len(src) == len(tgt)
|
||||||
|
return (PASS if ok else FAIL,
|
||||||
|
f"source={len(src)} target={len(tgt)}", [])
|
||||||
|
|
||||||
|
|
||||||
|
def check_paras(src, tgt):
|
||||||
|
s, t = count_paras(src), count_paras(tgt)
|
||||||
|
return (PASS if s == t else FAIL,
|
||||||
|
f"source={s} target={t}", [])
|
||||||
|
|
||||||
|
|
||||||
|
def check_headings(src, tgt):
|
||||||
|
s, t = heading_counts(src), heading_counts(tgt)
|
||||||
|
diffs = [f"H{i}: {s[i]} vs {t[i]}" for i in range(1, 7) if s[i] != t[i]]
|
||||||
|
return (PASS if not diffs else FAIL,
|
||||||
|
"; ".join(diffs) if diffs else "all levels match", diffs)
|
||||||
|
|
||||||
|
|
||||||
|
def check_emphasis(src, tgt):
|
||||||
|
"""D5 preservation: source emphasis must survive on the same line.
|
||||||
|
|
||||||
|
Target may add emphasis for titles/Sanskrit, so counts need not match.
|
||||||
|
"""
|
||||||
|
missing = []
|
||||||
|
for i, (s, t) in enumerate(zip(src, tgt), 1):
|
||||||
|
if EMPHASIS_RE.search(s) and not EMPHASIS_RE.search(t):
|
||||||
|
missing.append((i, s, t))
|
||||||
|
return (PASS if not missing else FAIL,
|
||||||
|
"all source emphasis preserved"
|
||||||
|
if not missing else
|
||||||
|
f"{len(missing)} source line(s) lost emphasis: "
|
||||||
|
+ ", ".join(f"S{i}" for i, _, _ in missing[:10]),
|
||||||
|
missing)
|
||||||
|
|
||||||
|
|
||||||
|
def check_comments(src, tgt):
|
||||||
|
s, t = len(COMMENT_RE.findall("\n".join(src))), len(COMMENT_RE.findall("\n".join(tgt)))
|
||||||
|
return (PASS if s == t else FAIL,
|
||||||
|
f"source={s} target={t}", [])
|
||||||
|
|
||||||
|
|
||||||
|
ANCHOR_RE = re.compile(r"\{#[^{}]*\}|\(\s*#?[^{}\n]*\)")
|
||||||
|
LINK_TGT_RE = re.compile(r"\[\d+\]\(\s*#")
|
||||||
|
|
||||||
|
|
||||||
|
def strip_structural(text):
|
||||||
|
"""Remove djot anchors {#...}, link destinations (...), and image paths —
|
||||||
|
structural markup that may legitimately contain Chinese."""
|
||||||
|
return ANCHOR_RE.sub("", text)
|
||||||
|
|
||||||
|
|
||||||
|
def check_cjk(tgt, allow=()):
|
||||||
|
bad = []
|
||||||
|
for i, line in enumerate(tgt, 1):
|
||||||
|
stripped = strip_structural(line)
|
||||||
|
for token in allow:
|
||||||
|
stripped = stripped.replace(token, "")
|
||||||
|
if CJK_RE.search(stripped):
|
||||||
|
bad.append((i, line))
|
||||||
|
return (PASS if not bad else FAIL,
|
||||||
|
"clean" if not bad else f"{len(bad)} line(s) contain CJK outside anchors/links: "
|
||||||
|
+ ", ".join(f"L{i}" for i, _ in bad[:10]),
|
||||||
|
bad)
|
||||||
|
|
||||||
|
|
||||||
|
def check_cn_punct(tgt):
|
||||||
|
bad = []
|
||||||
|
for i, line in enumerate(tgt, 1):
|
||||||
|
if CN_PUNCT_RE.search(line):
|
||||||
|
bad.append((i, line))
|
||||||
|
return (PASS if not bad else FAIL,
|
||||||
|
"clean" if not bad else f"{len(bad)} line(s) contain Chinese punctuation: "
|
||||||
|
+ ", ".join(f"L{i}" for i, _ in bad[:10]),
|
||||||
|
bad)
|
||||||
|
|
||||||
|
|
||||||
|
def check_bold(tgt):
|
||||||
|
bad = []
|
||||||
|
for i, line in enumerate(tgt, 1):
|
||||||
|
if BOLD_RE.search(line):
|
||||||
|
bad.append((i, line))
|
||||||
|
return (PASS if not bad else FAIL,
|
||||||
|
"clean" if not bad else f"{len(bad)} line(s) contain ** : "
|
||||||
|
+ ", ".join(f"L{i}" for i, _ in bad[:10]),
|
||||||
|
bad)
|
||||||
|
|
||||||
|
|
||||||
|
_ONES = ["", "one", "two", "three", "four", "five", "six", "seven", "eight",
|
||||||
|
"nine", "ten", "eleven", "twelve", "thirteen", "fourteen", "fifteen",
|
||||||
|
"sixteen", "seventeen", "eighteen", "nineteen"]
|
||||||
|
_TENS = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy",
|
||||||
|
"eighty", "ninety"]
|
||||||
|
|
||||||
|
|
||||||
|
def number_to_words(n):
|
||||||
|
if n < 20:
|
||||||
|
return _ONES[n]
|
||||||
|
if n < 100:
|
||||||
|
return (_TENS[n // 10] + ("-" + _ONES[n % 10] if n % 10 else ""))
|
||||||
|
if n < 1000:
|
||||||
|
return _ONES[n // 100] + " hundred" + (
|
||||||
|
(" " + number_to_words(n % 100)) if n % 100 else "")
|
||||||
|
if n < 1000000:
|
||||||
|
return number_to_words(n // 1000) + " thousand" + (
|
||||||
|
(" " + number_to_words(n % 1000)) if n % 1000 else "")
|
||||||
|
return number_to_words(n // 1000000) + " million" + (
|
||||||
|
(" " + number_to_words(n % 1000000)) if n % 1000000 else "")
|
||||||
|
|
||||||
|
|
||||||
|
def accepted_number_spellings(n, unit):
|
||||||
|
"""All English spellings that legitimately render source number n.
|
||||||
|
|
||||||
|
unit: "" (plain), "万" (×10^4), or "亿" (×10^8). The target may keep the
|
||||||
|
digits ("1,200"), spell them ("twelve hundred"), or scale the unit
|
||||||
|
("13 million" for 1300万, "18 billion" for 180亿).
|
||||||
|
"""
|
||||||
|
cands = {str(n), number_to_words(n)}
|
||||||
|
if 100 <= n < 10000 and n % 100 == 0: # "twelve hundred"
|
||||||
|
cands.add(f"{n // 100} hundred")
|
||||||
|
cands.add(number_to_words(n // 100) + " hundred")
|
||||||
|
value = n * (10 ** 4 if unit == "万" else 10 ** 8 if unit == "亿" else 1)
|
||||||
|
if value != n:
|
||||||
|
cands.add(str(value))
|
||||||
|
cands.add(number_to_words(value))
|
||||||
|
for divisor, suffix in ((10 ** 9, "billion"), (10 ** 6, "million"),
|
||||||
|
(10 ** 3, "thousand")):
|
||||||
|
if value % divisor == 0 and value // divisor > 0:
|
||||||
|
cands.add(f"{value // divisor} {suffix}")
|
||||||
|
cands.add(number_to_words(value // divisor) + " " + suffix)
|
||||||
|
return cands
|
||||||
|
|
||||||
|
|
||||||
|
def source_content_nums(src_text):
|
||||||
|
"""(number, unit) pairs from the source, excluding TOC page numbers
|
||||||
|
([N](#...)) that the project convention intentionally drops."""
|
||||||
|
stripped = LINK_TGT_RE.sub("", src_text)
|
||||||
|
out = []
|
||||||
|
for m in re.finditer(r"\d+(?:\s*(?:多\s*)?[万亿])?", stripped):
|
||||||
|
tok = m.group(0)
|
||||||
|
unit = tok[-1] if tok[-1] in "万亿" else ""
|
||||||
|
out.append((int(re.sub(r"\D", "", tok)), unit))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def check_digits(src, tgt):
|
||||||
|
src_nums = source_content_nums("\n".join(src))
|
||||||
|
tgt_text = " ".join(tgt).lower().replace(",", "")
|
||||||
|
missing = []
|
||||||
|
for n, unit in src_nums:
|
||||||
|
if any(s.lower() in tgt_text
|
||||||
|
for s in accepted_number_spellings(n, unit)):
|
||||||
|
continue
|
||||||
|
missing.append(str(n) + unit)
|
||||||
|
return (PASS if not missing else FAIL,
|
||||||
|
"all present" if not missing else f"missing in target: {', '.join(missing)}",
|
||||||
|
missing)
|
||||||
|
|
||||||
|
|
||||||
|
def check_terminology(src, tgt, terms):
|
||||||
|
"""A3: source term present -> some allowed rendering present in target.
|
||||||
|
|
||||||
|
FAIL when no rendering is found at all; WARN when found but under-counted
|
||||||
|
(inflections, line wraps, or a genuine drift the reviewer should verify).
|
||||||
|
"""
|
||||||
|
if not terms:
|
||||||
|
return PASS, "no term map provided; skipped", []
|
||||||
|
src_text = "\n".join(src)
|
||||||
|
tgt_text = " ".join(tgt).lower()
|
||||||
|
fails, warns = [], []
|
||||||
|
checked = 0
|
||||||
|
for cn, renderings in sorted(terms.items()):
|
||||||
|
n = src_text.count(cn)
|
||||||
|
if n == 0:
|
||||||
|
continue
|
||||||
|
checked += 1
|
||||||
|
hits = sum(tgt_text.count(r.lower()) for r in renderings)
|
||||||
|
if hits == 0:
|
||||||
|
fails.append(f"{cn} ({n}× in source) — none of {renderings} found in target")
|
||||||
|
elif hits < n:
|
||||||
|
warns.append(f"{cn} ({n}× in source, {hits}× rendered) — verify")
|
||||||
|
if fails:
|
||||||
|
status, detail = FAIL, f"{checked} term(s) checked; " + "; ".join(fails)
|
||||||
|
elif warns:
|
||||||
|
status, detail = WARN, f"{checked} term(s) checked; " + "; ".join(warns)
|
||||||
|
else:
|
||||||
|
status, detail = PASS, f"{checked} term(s) checked; all consistent"
|
||||||
|
return status, detail, fails + warns
|
||||||
|
|
||||||
|
|
||||||
|
def check_bilingual(src, tgt, bilingual_path):
|
||||||
|
if bilingual_path is None or not Path(bilingual_path).exists():
|
||||||
|
return PASS, "no bilingual.dj present; skipped", []
|
||||||
|
actual = Path(bilingual_path).read_text(encoding="utf-8").splitlines()
|
||||||
|
expected = []
|
||||||
|
for s, t in zip(src, tgt):
|
||||||
|
if s == "":
|
||||||
|
expected.append("")
|
||||||
|
else:
|
||||||
|
expected.extend([s, t, ""])
|
||||||
|
if expected and expected[-1] != "":
|
||||||
|
expected.append("")
|
||||||
|
if actual == expected:
|
||||||
|
return PASS, f"{len(actual)} lines match a regeneration", []
|
||||||
|
return FAIL, f"stale: {Path(bilingual_path)} differs from source+target regeneration", []
|
||||||
|
|
||||||
|
|
||||||
|
def run_checks(src, tgt, bilingual=None, term_map=None, allow_cjk=()):
|
||||||
|
return [
|
||||||
|
("line-count parity", *check_line_count(src, tgt)),
|
||||||
|
("paragraph parity", *check_paras(src, tgt)),
|
||||||
|
("heading parity", *check_headings(src, tgt)),
|
||||||
|
("emphasis preservation", *check_emphasis(src, tgt)),
|
||||||
|
("comment parity", *check_comments(src, tgt)),
|
||||||
|
("CJK leakage", *check_cjk(tgt, allow_cjk)),
|
||||||
|
("Chinese punctuation", *check_cn_punct(tgt)),
|
||||||
|
("bold leakage", *check_bold(tgt)),
|
||||||
|
("digit fidelity", *check_digits(src, tgt)),
|
||||||
|
("terminology", *check_terminology(src, tgt, term_map)),
|
||||||
|
("bilingual freshness", *check_bilingual(src, tgt, bilingual)),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(description="Deterministic translation checks")
|
||||||
|
ap.add_argument("paths", nargs="+", help="book dir, or source.dj target.dj")
|
||||||
|
ap.add_argument("--bilingual", default=None, help="bilingual.dj to verify")
|
||||||
|
ap.add_argument("--term-map", default=None, help="term map file")
|
||||||
|
ap.add_argument("--allow-cjk", default="", help="comma-separated CJK whitelist")
|
||||||
|
ap.add_argument("--json", action="store_true")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if len(args.paths) == 1 and Path(args.paths[0]).is_dir():
|
||||||
|
d = Path(args.paths[0])
|
||||||
|
src, tgt = d / "source.dj", d / "target.dj"
|
||||||
|
bilingual = args.bilingual or d / "bilingual.dj"
|
||||||
|
term_map = args.term_map or d / "term-map.md"
|
||||||
|
label = str(d)
|
||||||
|
elif len(args.paths) == 2:
|
||||||
|
src, tgt = Path(args.paths[0]), Path(args.paths[1])
|
||||||
|
bilingual = Path(args.bilingual) if args.bilingual else None
|
||||||
|
term_map = Path(args.term_map) if args.term_map else None
|
||||||
|
label = f"{src} -> {tgt}"
|
||||||
|
else:
|
||||||
|
ap.error("pass a book directory, or source.dj target.dj")
|
||||||
|
|
||||||
|
if not src.exists() or not tgt.exists():
|
||||||
|
ap.error(f"missing source or target: {src} / {tgt}")
|
||||||
|
|
||||||
|
src_lines = read_lines(src)
|
||||||
|
tgt_lines = read_lines(tgt)
|
||||||
|
allow = [t for t in args.allow_cjk.split(",") if t.strip()]
|
||||||
|
terms = {}
|
||||||
|
if term_map and Path(term_map).exists():
|
||||||
|
terms = term_map_from_markdown(Path(term_map).read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
checks = run_checks(src_lines, tgt_lines, bilingual, terms, allow)
|
||||||
|
failed = [name for name, status, *_ in checks if status == FAIL]
|
||||||
|
warned = [name for name, status, *_ in checks if status == WARN]
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
print(json.dumps({
|
||||||
|
"target": label,
|
||||||
|
"passed": [c[0] for c in checks if c[1] == PASS],
|
||||||
|
"warned": warned,
|
||||||
|
"failed": failed,
|
||||||
|
"details": {c[0]: {"status": c[1], "detail": c[2]} for c in checks},
|
||||||
|
}, ensure_ascii=False, indent=2))
|
||||||
|
else:
|
||||||
|
print(f"check-translation.py — {label}\n")
|
||||||
|
for name, status, detail, *_ in checks:
|
||||||
|
print(f"[{status:4}] {name}: {detail}")
|
||||||
|
summary = "ALL CHECKS PASSED"
|
||||||
|
if warned:
|
||||||
|
summary = f"PASSED with warnings: {', '.join(warned)}"
|
||||||
|
if failed:
|
||||||
|
summary = f"FAILED: {', '.join(failed)}"
|
||||||
|
print(f"\n{summary}")
|
||||||
|
|
||||||
|
sys.exit(0 if not failed else 1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
---
|
---
|
||||||
name: chinese-text-normalize
|
name: mpi-chinese-text-normalize
|
||||||
description: Normalize Chinese markdown files — remove extraneous mid-sentence line breaks from fixed-width exports while preserving TOC structures, section headers, and intentional paragraph breaks.
|
description: Normalize Chinese markdown files by removing extraneous mid-sentence line breaks from fixed-width exports while preserving TOC structures, section headers, and intentional paragraph breaks. Do not use for English prose, wiki-link index files, or mixed CJK/English documents without manual review.
|
||||||
|
compatibility: Requires Python 3.9+ and uv. The script's shebang invokes `uv run --script`; it uses only the standard library.
|
||||||
---
|
---
|
||||||
|
|
||||||
When Chinese text has hard line breaks at a fixed width (~20-25 chars) — common in PDF exports, OCR output, or poorly-converted documents — use this skill to join them into flowing paragraphs.
|
When Chinese text has hard line breaks at a fixed width (~20-25 chars) — common in PDF exports, OCR output, or poorly-converted documents — use this skill to join them into flowing paragraphs.
|
||||||
@@ -13,7 +14,13 @@ When Chinese text has hard line breaks at a fixed width (~20-25 chars) — commo
|
|||||||
|
|
||||||
## Approach
|
## Approach
|
||||||
|
|
||||||
Run `scripts/normalize_breaks.py <directory>` — it processes all .md files in the directory.
|
Run the script directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
skills/mpi-chinese-text-normalize/scripts/normalize_breaks.py <directory>
|
||||||
|
```
|
||||||
|
|
||||||
|
The shebang invokes `uv run --script`. It processes all `.md` files in the directory.
|
||||||
|
|
||||||
The script handles three file patterns:
|
The script handles three file patterns:
|
||||||
|
|
||||||
|
|||||||
Regular → Executable
+11
-1
@@ -1,3 +1,9 @@
|
|||||||
|
#!/usr/bin/env -S uv run --script
|
||||||
|
# /// script
|
||||||
|
# requires-python = ">=3.9"
|
||||||
|
# dependencies = []
|
||||||
|
# ///
|
||||||
|
|
||||||
"""
|
"""
|
||||||
Fix extraneous line breaks in Chinese markdown files.
|
Fix extraneous line breaks in Chinese markdown files.
|
||||||
|
|
||||||
@@ -6,8 +12,9 @@ Three file patterns:
|
|||||||
2. Mostly-paragraph with stray breaks + outline TOC -> join broken lines, preserve list items
|
2. Mostly-paragraph with stray breaks + outline TOC -> join broken lines, preserve list items
|
||||||
3. Already fine -> skip (idempotent)
|
3. Already fine -> skip (idempotent)
|
||||||
|
|
||||||
Usage: python3 normalize_breaks.py <directory>
|
Usage: ./normalize_breaks.py <directory>
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import re
|
import re
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -155,6 +162,9 @@ def process_file(filepath):
|
|||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
if len(sys.argv) != 2:
|
||||||
|
print("Usage: ./normalize_breaks.py <directory>", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
workdir = Path(sys.argv[1])
|
workdir = Path(sys.argv[1])
|
||||||
files = sorted(workdir.glob('*.md'))
|
files = sorted(workdir.glob('*.md'))
|
||||||
|
|
||||||
|
|||||||
@@ -1,253 +1,45 @@
|
|||||||
---
|
---
|
||||||
name: pdf-to-docx-conversion
|
name: mpi-pdf-to-docx-conversion
|
||||||
description: "Convert flowing text PDFs (Chinese or multi-language) to DOCX with proper fonts, styles, native bullets, lists, and embedded images. Preserves visual hierarchy from PDF font/size/color data."
|
description: Convert flowing text PDFs (Chinese or multi-language) to DOCX with proper fonts, styles, native bullets, lists, and embedded images. Use for text-based PDFs. Do not use for scanned/image PDFs, form-heavy PDFs, or documents where exact page layout must be preserved.
|
||||||
version: 1.0.0
|
|
||||||
author: Claude
|
|
||||||
license: MIT
|
license: MIT
|
||||||
platforms: [linux, macos, windows]
|
compatibility: Requires Python 3.9+ and uv. The script's shebang invokes `uv run --script`; dependencies (pymupdf, python-docx) are declared in the `/// script` metadata.
|
||||||
metadata:
|
|
||||||
hermes:
|
|
||||||
tags: [PDF, DOCX, Documents, python-docx, pymupdf]
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# PDF-to-DOCX Conversion
|
# PDF-to-DOCX Conversion
|
||||||
|
|
||||||
Convert PDF documents (especially flowing text documents in any language, including CJK) into well-structured DOCX files that preserve fonts, sizes, colors, and layout intent.
|
Convert text-based PDF documents (including CJK) into structured DOCX files that preserve fonts, sizes, colors, and layout intent.
|
||||||
|
|
||||||
## Prerequisites
|
## Quick start
|
||||||
|
|
||||||
|
For most PDFs, run the bundled converter directly:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install pymupdf python-docx
|
skills/mpi-pdf-to-docx-conversion/scripts/convert_pdf_to_docx.py input.pdf output.docx
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The shebang invokes `uv run --script`, which reads the `/// script` metadata block and installs `pymupdf` and `python-docx` automatically.
|
||||||
|
|
||||||
|
For documents with unusual fonts or structure, inspect first and pass a config dict. See `references/config-patterns.md` for the config schema and common patterns.
|
||||||
|
|
||||||
|
## Workflow
|
||||||
|
|
||||||
|
1. **Inspect the PDF.** Dump fonts, sizes, bullets, and header hierarchy. See `references/inspection-guide.md`.
|
||||||
|
2. **Configure.** Build a config dict matching the PDF's patterns (fonts, bullet fonts, skip fonts, numbered patterns, header sizes). See `references/config-patterns.md`.
|
||||||
|
3. **Convert.** Run `scripts/convert_pdf_to_docx.py` or import `PDFToDOCXConverter` in Python.
|
||||||
|
4. **Verify.** Check paragraph count, styles, bullets, images, and page-number leakage. See the checklist below.
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Style-aware**: reads actual font, size, bold, color from PDF spans
|
- Style-aware extraction (font, size, bold, color)
|
||||||
- **Native bullets**: uses Word `List Bullet` style instead of Wingdings glyphs
|
- Native Word bullets and numbering
|
||||||
- **Native numbering**: uses numbered list style for sequential items
|
- Image extraction and embedding
|
||||||
- **Image extraction**: detects and embeds PDF images into the DOCX
|
- Verse/poetry line splitting
|
||||||
- **Verse/poetry handling**: splits merged verse lines at semantic boundaries
|
- Multi-language support (CJK, RTL, mixed scripts)
|
||||||
- **Multi-language**: works with CJK, RTL, and mixed-script documents
|
- Flowing text across pages (no forced page breaks)
|
||||||
- **Flowing text**: text flows across pages; no forced page breaks
|
|
||||||
|
|
||||||
## Quick Start
|
## Verification checklist
|
||||||
|
|
||||||
```python
|
After conversion, inspect the DOCX:
|
||||||
from pdf_to_docx import convert_pdf_to_docx
|
|
||||||
|
|
||||||
convert_pdf_to_docx("input.pdf", "output.docx")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step-by-Step Workflow
|
|
||||||
|
|
||||||
### 1. Inspect the PDF
|
|
||||||
|
|
||||||
First, dump the PDF to understand its structure:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 << 'PY'
|
|
||||||
import pymupdf
|
|
||||||
doc = pymupdf.open("input.pdf")
|
|
||||||
for pi in range(len(doc)):
|
|
||||||
page = doc[pi]
|
|
||||||
blocks = page.get_text("dict")["blocks"]
|
|
||||||
for block in blocks:
|
|
||||||
if block["type"] != 0: continue
|
|
||||||
for line in block["lines"]:
|
|
||||||
for span in line["spans"]:
|
|
||||||
bbox = span["bbox"]
|
|
||||||
flags = span["flags"]
|
|
||||||
attrs = []
|
|
||||||
if flags & 2**1: attrs.append("I")
|
|
||||||
if flags & 2**4: attrs.append("B")
|
|
||||||
print(f" Y={bbox[1]:.0f} [{span['size']:.1f}pt {'+'.join(attrs) or '-'}] {span['font']} | {span['text']}")
|
|
||||||
PY
|
|
||||||
```
|
|
||||||
|
|
||||||
Key things to identify:
|
|
||||||
- **Fonts used** (map to DOCX fonts)
|
|
||||||
- **Bullet mechanism** (Wingdings? Unicode?)
|
|
||||||
- **Header hierarchy** (what size = section header vs sub-header)
|
|
||||||
- **Numbered lists** (what delimiter: `1)` `1.` `1)`)
|
|
||||||
- **Images** (check `page.get_images()`)
|
|
||||||
- **Special sections** (tables, verses, forms)
|
|
||||||
|
|
||||||
### 2. Configure the Converter
|
|
||||||
|
|
||||||
Create a config dict matching your PDF's patterns:
|
|
||||||
|
|
||||||
```python
|
|
||||||
config = {
|
|
||||||
"fonts": {
|
|
||||||
"title": "STHeitiSC-Medium",
|
|
||||||
"body": "HYShuSongErKW",
|
|
||||||
"page_number": "HelveticaNeue",
|
|
||||||
},
|
|
||||||
"header_sizes": {"section": 18, "sub": 15},
|
|
||||||
"body_size": 12,
|
|
||||||
"bullet_fonts": ["Wingdings", "Wingdings 2", "Wingdings 3"],
|
|
||||||
"page_number_font": "HelveticaNeue",
|
|
||||||
"numbered_patterns": [r'^\d+\)', r'^\d+\.'], # detect numbered items
|
|
||||||
"skip_fonts": ["HelveticaNeue"], # fonts to skip (page numbers)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Run the Conversion
|
|
||||||
|
|
||||||
```python
|
|
||||||
from pdf_to_docx import PDFToDOCXConverter
|
|
||||||
|
|
||||||
converter = PDFToDOCXConverter(config)
|
|
||||||
converter.convert("input.pdf", "output.docx")
|
|
||||||
```
|
|
||||||
|
|
||||||
## Core Classes
|
|
||||||
|
|
||||||
### PDFToDOCXConverter
|
|
||||||
|
|
||||||
```python
|
|
||||||
class PDFToDOCXConverter:
|
|
||||||
def __init__(self, config=None):
|
|
||||||
self.cfg = config or self._default_config()
|
|
||||||
|
|
||||||
def convert(self, pdf_path: str, docx_path: str):
|
|
||||||
"""Main entry point."""
|
|
||||||
# 1. Extract all spans
|
|
||||||
# 2. Merge Wingdings bullets with body text
|
|
||||||
# 3. Group lines into paragraphs by Y-gap and style changes
|
|
||||||
# 4. Post-process: split compact lists, verses, merged steps
|
|
||||||
# 5. Build DOCX with proper styles
|
|
||||||
# 6. Embed images
|
|
||||||
pass
|
|
||||||
|
|
||||||
def extract_spans(self, pdf_path: str) -> list[dict]:
|
|
||||||
"""Extract all text spans with full style info."""
|
|
||||||
doc = pymupdf.open(pdf_path)
|
|
||||||
raw = []
|
|
||||||
for pi in range(len(doc)):
|
|
||||||
for block in doc[pi].get_text("dict")["blocks"]:
|
|
||||||
if block["type"] != 0: continue
|
|
||||||
for line in block["lines"]:
|
|
||||||
for span in line["spans"]:
|
|
||||||
raw.append({
|
|
||||||
"text": span["text"], "font": span["font"],
|
|
||||||
"size": span["size"], "flags": span["flags"],
|
|
||||||
"color": span["color"], "bbox": span["bbox"],
|
|
||||||
})
|
|
||||||
return raw
|
|
||||||
|
|
||||||
def detect_bullets(self, line: list) -> bool:
|
|
||||||
"""Check if first span in line is a Wingdings bullet."""
|
|
||||||
return "Wingdings" in line[0]["font"]
|
|
||||||
|
|
||||||
def group_paragraphs(self, lines: list) -> list[dict]:
|
|
||||||
"""Group raw lines into logical paragraphs."""
|
|
||||||
# Group by Y-gap threshold (typically 20-30pt)
|
|
||||||
# Break on style change (font change, size > threshold, bold toggle)
|
|
||||||
# Break on attribution lines (——节选自...)
|
|
||||||
# Break on special fonts (STHeitiSC-Light etc.)
|
|
||||||
pass
|
|
||||||
|
|
||||||
def post_process(self, paras: list) -> list[dict]:
|
|
||||||
"""Split merged compact lists and verses."""
|
|
||||||
# See examples below for common patterns
|
|
||||||
pass
|
|
||||||
```
|
|
||||||
|
|
||||||
## Common Post-Processing Patterns
|
|
||||||
|
|
||||||
### Compact Numbered Lists
|
|
||||||
|
|
||||||
When the PDF flows list items together in one paragraph:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def split_compact_list(text: str) -> list[str]:
|
|
||||||
"""Split '1) foo 2) bar 3) baz' into separate items."""
|
|
||||||
parts = re.split(r'(?=\d+\))', text)
|
|
||||||
return [p for p in parts if p.strip()]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Verse / Poetry Lines
|
|
||||||
|
|
||||||
When the PDF merges verse lines that should be on separate lines:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def split_verse(text: str, split_markers: list[str]) -> list[str]:
|
|
||||||
"""Split verse at semantic phrase boundaries.
|
|
||||||
Example markers: ['感恩', '愿', '更愿']"""
|
|
||||||
pattern = '|'.join(f'(?={m})' for m in split_markers)
|
|
||||||
return [p for p in re.split(pattern, text) if p.strip()]
|
|
||||||
```
|
|
||||||
|
|
||||||
### 小组交流流程 / Process Steps
|
|
||||||
|
|
||||||
Split Chinese process steps numbered 一、二、三、etc.:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def split_chinese_steps(text: str) -> list[str]:
|
|
||||||
"""Split '一、foo 二、bar' into separate items."""
|
|
||||||
parts = re.split(r'(?=[一二三四五六七八九十]、)', text)
|
|
||||||
return [p.strip() for p in parts if p.strip()]
|
|
||||||
```
|
|
||||||
|
|
||||||
## DOCX Construction
|
|
||||||
|
|
||||||
### Font Setup (East Asian fonts)
|
|
||||||
|
|
||||||
```python
|
|
||||||
from docx.oxml.ns import qn
|
|
||||||
from docx.oxml import OxmlElement
|
|
||||||
|
|
||||||
def set_east_asian_font(run, name: str):
|
|
||||||
"""Set CJK font properly in python-docx."""
|
|
||||||
rPr = run._element.get_or_add_rPr()
|
|
||||||
rFonts = rPr.find(qn('w:rFonts'))
|
|
||||||
if rFonts is None:
|
|
||||||
rFonts = OxmlElement('w:rFonts')
|
|
||||||
rPr.insert(0, rFonts)
|
|
||||||
rFonts.set(qn('w:eastAsia'), name)
|
|
||||||
rFonts.set(qn('w:ascii'), name)
|
|
||||||
rFonts.set(qn('w:hAnsi'), name)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Bullet Items
|
|
||||||
|
|
||||||
Use native Word bullets, NOT Wingdings characters:
|
|
||||||
|
|
||||||
```python
|
|
||||||
p = doc.add_paragraph(style='List Bullet')
|
|
||||||
p.clear()
|
|
||||||
run = p.add_run("Your bullet text here")
|
|
||||||
set_east_asian_font(run, font_name)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Image Embedding
|
|
||||||
|
|
||||||
```python
|
|
||||||
from docx.shared import Inches
|
|
||||||
|
|
||||||
p = doc.add_paragraph()
|
|
||||||
p.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
|
||||||
run = p.add_run()
|
|
||||||
run.add_picture(image_path, width=Inches(3.5))
|
|
||||||
```
|
|
||||||
|
|
||||||
Extract images from PDF first:
|
|
||||||
|
|
||||||
```python
|
|
||||||
doc = pymupdf.open("input.pdf")
|
|
||||||
for pi in range(len(doc)):
|
|
||||||
images = doc[pi].get_images()
|
|
||||||
for idx, img in enumerate(images):
|
|
||||||
xref = img[0]
|
|
||||||
base = doc.extract_image(xref)
|
|
||||||
with open(f"extracted_{pi}_{idx}.{base['ext']}", 'wb') as f:
|
|
||||||
f.write(base['image'])
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verification Checklist
|
|
||||||
|
|
||||||
After conversion, verify the DOCX:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
python3 -c "
|
python3 -c "
|
||||||
@@ -256,67 +48,33 @@ doc = Document('output.docx')
|
|||||||
print(f'Paragraphs: {len(doc.paragraphs)}')
|
print(f'Paragraphs: {len(doc.paragraphs)}')
|
||||||
for i, p in enumerate(doc.paragraphs):
|
for i, p in enumerate(doc.paragraphs):
|
||||||
style = p.style.name if p.style else '-'
|
style = p.style.name if p.style else '-'
|
||||||
txt = p.text[:80]
|
print(f'[{i:2d}] [{style:15s}] {p.text[:80]}')
|
||||||
print(f'[{i:2d}] [{style:15s}] {txt}')
|
|
||||||
"
|
"
|
||||||
```
|
```
|
||||||
|
|
||||||
Check for:
|
Check for:
|
||||||
1. [ ] All sections present (count paragraphs)
|
1. [ ] All sections present (paragraph count matches expectations)
|
||||||
2. [ ] No merged verses or lists
|
2. [ ] No merged verses or lists
|
||||||
3. [ ] Headers are bold + larger size
|
3. [ ] Headers are bold and larger than body text
|
||||||
4. [ ] Bullets use `List Bullet` style
|
4. [ ] Bullets use the `List Bullet` style
|
||||||
5. [ ] Numbered items are separate paragraphs
|
5. [ ] Numbered items are separate paragraphs
|
||||||
6. [ ] Images present in `word/media/`
|
6. [ ] Images are embedded in `word/media/`
|
||||||
7. [ ] Attribution lines right-aligned/indented
|
7. [ ] Attribution lines are right-aligned or indented
|
||||||
8. [ ] No page numbers leaked into body
|
8. [ ] Page numbers are not leaked into body text
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
| Symptom | Cause | Fix |
|
If output is wrong, see `references/troubleshooting.md` for a full symptom/cause/fix table. Common first checks:
|
||||||
|---------|-------|-----|
|
|
||||||
| Text over-merged | Y-gap threshold too high | Lower gap threshold (e.g. 20 → 15) |
|
|
||||||
| Missing sections | Skipped by font filter | Add font to skip_fonts or remove filter |
|
|
||||||
| Over-split lines | Y-gap threshold too low | Raise gap threshold (e.g. 20 → 30) |
|
|
||||||
| Wingdings boxes | Unicode bullet inserted | Use `style='List Bullet'` instead |
|
|
||||||
| CJK font wrong | East Asian font not set | Use `set_east_asian_font()` helper |
|
|
||||||
| Image missing | Not extracted before DOCX build | Run `extract_images()` first |
|
|
||||||
| Verse mangled | Regex too aggressive | Tune verse splitting pattern |
|
|
||||||
|
|
||||||
## Full Example Script
|
- Over-merged text → lower the Y-gap threshold.
|
||||||
|
- Over-split lines → raise the Y-gap threshold.
|
||||||
|
- Wingdings boxes → use native `List Bullet` style instead.
|
||||||
|
- Missing images → ensure images are extracted before DOCX construction.
|
||||||
|
|
||||||
See `scripts/convert_pdf_to_docx.py` for a production-ready converter with all patterns pre-configured.
|
## References
|
||||||
|
|
||||||
```python
|
- `references/inspection-guide.md` — dump PDF structure and interpret spans
|
||||||
# scripts/convert_pdf_to_docx.py
|
- `references/config-patterns.md` — config dict, compact lists, verses, numbered steps
|
||||||
import pymupdf, re
|
- `references/docx-construction.md` — East Asian fonts, bullets, image embedding
|
||||||
from docx import Document
|
- `references/troubleshooting.md` — symptom/cause/fix table
|
||||||
from docx.shared import Pt, Cm, Inches, RGBColor
|
- `scripts/convert_pdf_to_docx.py` — production-ready converter
|
||||||
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
|
||||||
from docx.oxml.ns import qn
|
|
||||||
from docx.oxml import OxmlElement
|
|
||||||
|
|
||||||
def convert_pdf_to_docx(pdf_path: str, docx_path: str, config: dict = None):
|
|
||||||
"""Convert a flowing text PDF to DOCX."""
|
|
||||||
cfg = config or {}
|
|
||||||
|
|
||||||
# Extract
|
|
||||||
doc_pdf = pymupdf.open(pdf_path)
|
|
||||||
raw = []
|
|
||||||
for pi in range(len(doc_pdf)):
|
|
||||||
for block in doc_pdf[pi].get_text("dict")["blocks"]:
|
|
||||||
if block["type"] != 0: continue
|
|
||||||
for line in block["lines"]:
|
|
||||||
ss = line["spans"]
|
|
||||||
is_b = any("Wingdings" in s["font"] for s in ss[:1])
|
|
||||||
text = "".join(s["text"] for s in ss)
|
|
||||||
dom = ss[1] if (is_b and len(ss) > 1) else ss[0]
|
|
||||||
raw.append(dict(text=text, font=dom["font"], size=dom["size"],
|
|
||||||
bold=bool(dom["flags"] & 2**4), color=dom["color"],
|
|
||||||
x=dom["bbox"][0], y=dom["bbox"][1], is_bullet=is_b))
|
|
||||||
|
|
||||||
# ... (merge bullets, group paragraphs, post-process, build DOCX)
|
|
||||||
|
|
||||||
# Run:
|
|
||||||
# python scripts/convert_pdf_to_docx.py input.pdf output.docx
|
|
||||||
```
|
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# PDF-to-DOCX Config Patterns
|
||||||
|
|
||||||
|
The converter accepts a config dict that maps PDF-specific patterns to DOCX behavior.
|
||||||
|
|
||||||
|
## Example config
|
||||||
|
|
||||||
|
```python
|
||||||
|
config = {
|
||||||
|
"fonts": {
|
||||||
|
"title": "STHeitiSC-Medium",
|
||||||
|
"body": "HYShuSongErKW",
|
||||||
|
"page_number": "HelveticaNeue",
|
||||||
|
},
|
||||||
|
"header_sizes": {"section": 18, "sub": 15},
|
||||||
|
"body_size": 12,
|
||||||
|
"bullet_fonts": ["Wingdings", "Wingdings 2", "Wingdings 3"],
|
||||||
|
"page_number_font": "HelveticaNeue",
|
||||||
|
"numbered_patterns": [r'^\d+\)', r'^\d+\.'],
|
||||||
|
"skip_fonts": ["HelveticaNeue"],
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common post-processing patterns
|
||||||
|
|
||||||
|
### Compact numbered lists
|
||||||
|
|
||||||
|
Split items that flowed together in one PDF paragraph:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def split_compact_list(text: str) -> list[str]:
|
||||||
|
parts = re.split(r'(?=\d+\))', text)
|
||||||
|
return [p for p in parts if p.strip()]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verse / poetry lines
|
||||||
|
|
||||||
|
Split merged verse at semantic phrase boundaries:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def split_verse(text: str, split_markers: list[str]) -> list[str]:
|
||||||
|
pattern = '|'.join(f'(?={m})' for m in split_markers)
|
||||||
|
return [p for p in re.split(pattern, text) if p.strip()]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Chinese process steps
|
||||||
|
|
||||||
|
Split `一、foo 二、bar` into separate items:
|
||||||
|
|
||||||
|
```python
|
||||||
|
def split_chinese_steps(text: str) -> list[str]:
|
||||||
|
parts = re.split(r'(?=[一二三四五六七八九十]、)', text)
|
||||||
|
return [p.strip() for p in parts if p.strip()]
|
||||||
|
```
|
||||||
|
|
||||||
|
## Tuning thresholds
|
||||||
|
|
||||||
|
- **Y-gap threshold** (typically 20–30 pt): controls how aggressively lines are grouped into paragraphs.
|
||||||
|
- **Style-change threshold**: break paragraphs on font change, size jump, or bold toggle when the difference exceeds this value.
|
||||||
|
|
||||||
|
See `references/troubleshooting.md` for threshold adjustment guidance.
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# DOCX Construction Notes
|
||||||
|
|
||||||
|
Low-level notes for building the DOCX output with python-docx.
|
||||||
|
|
||||||
|
## East Asian fonts
|
||||||
|
|
||||||
|
Set CJK fonts properly on a run:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from docx.oxml.ns import qn
|
||||||
|
from docx.oxml import OxmlElement
|
||||||
|
|
||||||
|
def set_east_asian_font(run, name: str):
|
||||||
|
rPr = run._element.get_or_add_rPr()
|
||||||
|
rFonts = rPr.find(qn('w:rFonts'))
|
||||||
|
if rFonts is None:
|
||||||
|
rFonts = OxmlElement('w:rFonts')
|
||||||
|
rPr.insert(0, rFonts)
|
||||||
|
rFonts.set(qn('w:eastAsia'), name)
|
||||||
|
rFonts.set(qn('w:ascii'), name)
|
||||||
|
rFonts.set(qn('w:hAnsi'), name)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Native bullets
|
||||||
|
|
||||||
|
Use Word's `List Bullet` style, not Wingdings characters:
|
||||||
|
|
||||||
|
```python
|
||||||
|
p = doc.add_paragraph(style='List Bullet')
|
||||||
|
p.clear()
|
||||||
|
run = p.add_run("Bullet text")
|
||||||
|
set_east_asian_font(run, font_name)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Image embedding
|
||||||
|
|
||||||
|
Extract images from the PDF first, then embed them:
|
||||||
|
|
||||||
|
```python
|
||||||
|
from docx.shared import Inches
|
||||||
|
|
||||||
|
# Extract
|
||||||
|
doc = pymupdf.open("input.pdf")
|
||||||
|
for pi in range(len(doc)):
|
||||||
|
for idx, img in enumerate(doc[pi].get_images()):
|
||||||
|
xref = img[0]
|
||||||
|
base = doc.extract_image(xref)
|
||||||
|
path = f"extracted_{pi}_{idx}.{base['ext']}"
|
||||||
|
with open(path, 'wb') as f:
|
||||||
|
f.write(base['image'])
|
||||||
|
|
||||||
|
# Embed
|
||||||
|
p = doc.add_paragraph()
|
||||||
|
p.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
||||||
|
run = p.add_run()
|
||||||
|
run.add_picture(image_path, width=Inches(3.5))
|
||||||
|
```
|
||||||
|
|
||||||
|
See `scripts/convert_pdf_to_docx.py` for the production-ready implementation.
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
# PDF Inspection Guide
|
||||||
|
|
||||||
|
Before converting a PDF, dump its text spans to understand fonts, sizes, bullets, and structure.
|
||||||
|
|
||||||
|
## Dump spans
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 << 'PY'
|
||||||
|
import pymupdf
|
||||||
|
doc = pymupdf.open("input.pdf")
|
||||||
|
for pi in range(len(doc)):
|
||||||
|
page = doc[pi]
|
||||||
|
blocks = page.get_text("dict")["blocks"]
|
||||||
|
for block in blocks:
|
||||||
|
if block["type"] != 0: continue
|
||||||
|
for line in block["lines"]:
|
||||||
|
for span in line["spans"]:
|
||||||
|
bbox = span["bbox"]
|
||||||
|
flags = span["flags"]
|
||||||
|
attrs = []
|
||||||
|
if flags & 2**1: attrs.append("I")
|
||||||
|
if flags & 2**4: attrs.append("B")
|
||||||
|
print(f" Y={bbox[1]:.0f} [{span['size']:.1f}pt {'+'.join(attrs) or '-'}] {span['font']} | {span['text']}")
|
||||||
|
PY
|
||||||
|
```
|
||||||
|
|
||||||
|
## What to identify
|
||||||
|
|
||||||
|
- **Fonts used** — map PDF font names to DOCX font names.
|
||||||
|
- **Bullet mechanism** — Wingdings glyphs, Unicode bullets, or something else.
|
||||||
|
- **Header hierarchy** — which font size marks section vs. sub-section headers.
|
||||||
|
- **Numbered lists** — delimiter style: `1)`, `1.`, `1)`, `一、`.
|
||||||
|
- **Images** — check `page.get_images()` on each page.
|
||||||
|
- **Special sections** — tables, verses, attribution lines, page numbers, forms.
|
||||||
|
|
||||||
|
## Page numbers
|
||||||
|
|
||||||
|
Page-number fonts are usually small and repeated on every page. Add them to `skip_fonts` in the config so they do not leak into body text.
|
||||||
|
|
||||||
|
## Attribution lines
|
||||||
|
|
||||||
|
Lines like `——节选自...` often use a different font or indentation. The converter breaks paragraphs on these; verify the break point after conversion.
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
# PDF-to-DOCX Troubleshooting
|
||||||
|
|
||||||
|
| Symptom | Cause | Fix |
|
||||||
|
|---------|-------|-----|
|
||||||
|
| Text over-merged | Y-gap threshold too high | Lower gap threshold (e.g. 20 → 15) |
|
||||||
|
| Missing sections | Skipped by font filter | Add font to `skip_fonts` or remove filter |
|
||||||
|
| Over-split lines | Y-gap threshold too low | Raise gap threshold (e.g. 20 → 30) |
|
||||||
|
| Wingdings boxes | Unicode bullet inserted | Use `style='List Bullet'` instead |
|
||||||
|
| CJK font wrong | East Asian font not set | Use `set_east_asian_font()` helper |
|
||||||
|
| Image missing | Not extracted before DOCX build | Run image extraction first |
|
||||||
|
| Verse mangled | Regex too aggressive | Tune verse splitting pattern |
|
||||||
|
|
||||||
|
## General debugging steps
|
||||||
|
|
||||||
|
1. Re-inspect the PDF with the span dump from `references/inspection-guide.md`.
|
||||||
|
2. Compare the config against the actual fonts and sizes in the dump.
|
||||||
|
3. Run the verification script in `SKILL.md` and check paragraph styles.
|
||||||
|
4. Adjust one threshold at a time and re-convert.
|
||||||
Regular → Executable
+12
-4
@@ -1,13 +1,21 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env -S uv run --script
|
||||||
|
# /// script
|
||||||
|
# requires-python = ">=3.9"
|
||||||
|
# dependencies = [
|
||||||
|
# "pymupdf",
|
||||||
|
# "python-docx",
|
||||||
|
# ]
|
||||||
|
# ///
|
||||||
|
|
||||||
"""
|
"""
|
||||||
Production-ready PDF-to-DOCX converter.
|
Production-ready PDF-to-DOCX converter.
|
||||||
Handles multi-language text, mixed fonts, bullets, numbered lists, verses,
|
Handles multi-language text, mixed fonts, bullets, numbered lists, verses,
|
||||||
attributions, images, and flowing text across pages.
|
attributions, images, and flowing text across pages.
|
||||||
|
|
||||||
Usage:
|
Usage:
|
||||||
python convert_pdf_to_docx.py input.pdf output.docx
|
./convert_pdf_to_docx.py input.pdf output.docx
|
||||||
|
|
||||||
Requires: pip install pymupdf python-docx
|
Requires: uv (dependencies are declared in the /// script block above)
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import sys
|
import sys
|
||||||
@@ -544,6 +552,6 @@ def convert_pdf_to_docx(pdf_path: str, docx_path: str, config: dict = None):
|
|||||||
# CLI
|
# CLI
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
if len(sys.argv) != 3:
|
if len(sys.argv) != 3:
|
||||||
print("Usage: python convert_pdf_to_docx.py <input.pdf> <output.docx>")
|
print("Usage: ./convert_pdf_to_docx.py <input.pdf> <output.docx>")
|
||||||
sys.exit(1)
|
sys.exit(1)
|
||||||
convert_pdf_to_docx(sys.argv[1], sys.argv[2])
|
convert_pdf_to_docx(sys.argv[1], sys.argv[2])
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
---
|
---
|
||||||
name: pptx-translate
|
name: mpi-pptx-translate
|
||||||
description: Translate PowerPoint files between Chinese and English — extract strings to YAML, translate, quality review, and write back with font-shrink + auto-fit for layout.
|
description: Translate PowerPoint files between Chinese and English — extract strings to YAML, translate, quality review, and write back with font-shrink + auto-fit for layout. Use only for .pptx files. Do not use for .ppt, Google Slides exports, or PDFs.
|
||||||
category: productivity
|
category: productivity
|
||||||
|
compatibility: Requires Python 3.9+ and uv. The scripts' shebang invokes `uv run --script`; dependencies (python-pptx, pyyaml) are declared in the `/// script` metadata.
|
||||||
---
|
---
|
||||||
|
|
||||||
# PPTX Translation
|
# PPTX Translation
|
||||||
@@ -12,7 +13,13 @@ Translate `.pptx` files between Chinese and English. Covers the full pipeline: e
|
|||||||
|
|
||||||
### 1. Extract strings to YAML
|
### 1. Extract strings to YAML
|
||||||
|
|
||||||
Run `toolkit/scripts/extract.py original.pptx strings.yaml`. Produces YAML with entries:
|
Run the script directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
skills/mpi-pptx-translate/scripts/extract.py original.pptx strings.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
The shebang invokes `uv run --script`, which reads the `/// script` metadata block and installs `python-pptx` and `pyyaml` automatically. Produces YAML with entries:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
- slide: 1
|
- slide: 1
|
||||||
@@ -57,7 +64,9 @@ Scan for:
|
|||||||
|
|
||||||
### 4. Write back with layout fixes
|
### 4. Write back with layout fixes
|
||||||
|
|
||||||
Run `toolkit/scripts/build.py strings.yaml original.pptx translated.pptx`.
|
```bash
|
||||||
|
skills/mpi-pptx-translate/scripts/build.py strings.yaml original.pptx translated.pptx
|
||||||
|
```
|
||||||
|
|
||||||
The script:
|
The script:
|
||||||
- Replaces text in matching paragraphs (clears all runs, sets first run)
|
- Replaces text in matching paragraphs (clears all runs, sets first run)
|
||||||
@@ -80,5 +89,5 @@ The absorbed `pptx-translation` skill had alternate script names: `extract_pptx.
|
|||||||
|
|
||||||
## Scripts
|
## Scripts
|
||||||
|
|
||||||
- `toolkit/scripts/extract.py` — extract strings from PPTX to YAML
|
- `skills/mpi-pptx-translate/scripts/extract.py` — extract strings from PPTX to YAML
|
||||||
- `toolkit/scripts/build.py` — write translations back with font shrink + auto-fit
|
- `skills/mpi-pptx-translate/scripts/build.py` — write translations back with font shrink + auto-fit
|
||||||
|
|||||||
Regular → Executable
+24
-9
@@ -1,3 +1,12 @@
|
|||||||
|
#!/usr/bin/env -S uv run --script
|
||||||
|
# /// script
|
||||||
|
# requires-python = ">=3.9"
|
||||||
|
# dependencies = [
|
||||||
|
# "python-pptx",
|
||||||
|
# "pyyaml",
|
||||||
|
# ]
|
||||||
|
# ///
|
||||||
|
|
||||||
import sys, yaml
|
import sys, yaml
|
||||||
from pptx import Presentation
|
from pptx import Presentation
|
||||||
from pptx.util import Pt
|
from pptx.util import Pt
|
||||||
@@ -5,15 +14,6 @@ from pptx.enum.text import MSO_AUTO_SIZE
|
|||||||
|
|
||||||
FONT_SCALE = 0.82 # shrink ~18%
|
FONT_SCALE = 0.82 # shrink ~18%
|
||||||
|
|
||||||
yaml_path, src_path, out_path = sys.argv[1], sys.argv[2], sys.argv[3]
|
|
||||||
|
|
||||||
with open(yaml_path) as f:
|
|
||||||
entries = yaml.safe_load(f)
|
|
||||||
|
|
||||||
index = {}
|
|
||||||
for e in entries:
|
|
||||||
index[(e["slide"], e["shape"], e["run"])] = e["en"]
|
|
||||||
|
|
||||||
|
|
||||||
def shrink_font_tf(tf):
|
def shrink_font_tf(tf):
|
||||||
for para in tf.paragraphs:
|
for para in tf.paragraphs:
|
||||||
@@ -26,6 +26,14 @@ def shrink_font_tf(tf):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def build(yaml_path, src_path, out_path):
|
||||||
|
with open(yaml_path) as f:
|
||||||
|
entries = yaml.safe_load(f)
|
||||||
|
|
||||||
|
index = {}
|
||||||
|
for e in entries:
|
||||||
|
index[(e["slide"], e["shape"], e["run"])] = e["en"]
|
||||||
|
|
||||||
prs = Presentation(src_path)
|
prs = Presentation(src_path)
|
||||||
|
|
||||||
for slide_num, slide in enumerate(prs.slides, 1):
|
for slide_num, slide in enumerate(prs.slides, 1):
|
||||||
@@ -66,3 +74,10 @@ for slide_num, slide in enumerate(prs.slides, 1):
|
|||||||
|
|
||||||
prs.save(out_path)
|
prs.save(out_path)
|
||||||
print(f"Saved {out_path}")
|
print(f"Saved {out_path}")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
if len(sys.argv) != 4:
|
||||||
|
print("Usage: ./build.py <strings.yaml> <input.pptx> <output.pptx>", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
|
build(sys.argv[1], sys.argv[2], sys.argv[3])
|
||||||
|
|||||||
Regular → Executable
+12
@@ -1,3 +1,12 @@
|
|||||||
|
#!/usr/bin/env -S uv run --script
|
||||||
|
# /// script
|
||||||
|
# requires-python = ">=3.9"
|
||||||
|
# dependencies = [
|
||||||
|
# "python-pptx",
|
||||||
|
# "pyyaml",
|
||||||
|
# ]
|
||||||
|
# ///
|
||||||
|
|
||||||
import sys, yaml
|
import sys, yaml
|
||||||
from pptx import Presentation
|
from pptx import Presentation
|
||||||
|
|
||||||
@@ -62,6 +71,9 @@ def extract(pptx_path):
|
|||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
if len(sys.argv) != 3:
|
||||||
|
print("Usage: ./extract.py <input.pptx> <output.yaml>", file=sys.stderr)
|
||||||
|
sys.exit(1)
|
||||||
entries = extract(sys.argv[1])
|
entries = extract(sys.argv[1])
|
||||||
with open(sys.argv[2], "w") as f:
|
with open(sys.argv[2], "w") as f:
|
||||||
yaml.dump(entries, f, allow_unicode=True, default_flow_style=False, sort_keys=False)
|
yaml.dump(entries, f, allow_unicode=True, default_flow_style=False, sort_keys=False)
|
||||||
|
|||||||
@@ -1,19 +1,20 @@
|
|||||||
---
|
---
|
||||||
name: terms-search
|
name: mpi-terms-search
|
||||||
description: Full-text search across the MPI term database. Use when translating or looking up Chinese-English Buddhist/MPI terminology.
|
description: Full-text search across the MPI Buddhist/Dharma term database. Use when translating or reviewing Chinese-English Buddhist terminology. Do not use for general Chinese-English dictionary lookup outside MPI conventions.
|
||||||
category: research
|
category: research
|
||||||
|
compatibility: Requires Python 3; SQLite database is bundled.
|
||||||
---
|
---
|
||||||
|
|
||||||
# Terms Search
|
# Terms Search
|
||||||
|
|
||||||
Database: `$MPI_PROJECT_ROOT/toolkit/terms-database/termlib.duckdb`
|
Database: `toolkit/terms-database/termlib.sqlite` (SQLite)
|
||||||
CLI: `$MPI_PROJECT_ROOT/toolkit/terms-database/search.py`
|
CLI: `toolkit/terms-database/search.py`
|
||||||
Server: `$MPI_PROJECT_ROOT/toolkit/terms-database/server.py`
|
Server: `toolkit/terms-database/server.py`
|
||||||
|
|
||||||
## CLI (preferred)
|
## CLI (preferred)
|
||||||
|
|
||||||
```
|
```
|
||||||
$MPI_PROJECT_ROOT/toolkit/terms-database/search.py <query> [limit]
|
toolkit/terms-database/search.py <query> [limit]
|
||||||
```
|
```
|
||||||
|
|
||||||
Multi-word queries are ANDed. Searches both `zh` and `en` columns.
|
Multi-word queries are ANDed. Searches both `zh` and `en` columns.
|
||||||
@@ -22,7 +23,7 @@ Multi-word queries are ANDed. Searches both `zh` and `en` columns.
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
import sys
|
import sys
|
||||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
sys.path.insert(0, 'toolkit/terms-database')
|
||||||
from search import search
|
from search import search
|
||||||
results = search("空性", limit=5, src="DoT定稿")
|
results = search("空性", limit=5, src="DoT定稿")
|
||||||
# → list of {zh, en, loc, source} dicts
|
# → list of {zh, en, loc, source} dicts
|
||||||
@@ -32,7 +33,7 @@ Use this inside `execute_code` scripts for batch lookups — no subprocess neede
|
|||||||
|
|
||||||
## HTTP API (use only when CLI is insufficient)
|
## HTTP API (use only when CLI is insufficient)
|
||||||
|
|
||||||
Start: `python3 $MPI_PROJECT_ROOT/toolkit/terms-database/server.py` (port 8910)
|
Start: `python3 toolkit/terms-database/server.py` (port 8910)
|
||||||
|
|
||||||
- `GET /` — plain HTML UI (form + results table, no CSS)
|
- `GET /` — plain HTML UI (form + results table, no CSS)
|
||||||
- `GET /` — plain HTML UI (form + results table, no CSS)
|
- `GET /` — plain HTML UI (form + results table, no CSS)
|
||||||
@@ -61,20 +62,19 @@ Errors return `{"error": "..."}` with HTTP 500 (API) or shown inline (UI).
|
|||||||
| 禅意项目 | 14+11 | Zen program terms |
|
| 禅意项目 | 14+11 | Zen program terms |
|
||||||
| 公案 | 8 | Chan koans |
|
| 公案 | 8 | Chan koans |
|
||||||
|
|
||||||
## Direct DuckDB
|
## Direct SQLite
|
||||||
|
|
||||||
```
|
```
|
||||||
duckdb $MPI_PROJECT_ROOT/toolkit/terms-database/termlib.duckdb
|
sqlite3 toolkit/terms-database/termlib.sqlite
|
||||||
```
|
```
|
||||||
|
|
||||||
Key tables: `unified_terms_flat` (zh, en, loc, source), individual source tables, `unified_terms` view.
|
Key table: `terms` (zh, en, loc, source).
|
||||||
|
|
||||||
## Rebuilding
|
## Rebuilding
|
||||||
|
|
||||||
Terms data comes from `$MPI_PROJECT_ROOT/guide/03 术语库/`. To rebuild:
|
Terms data comes from `guide/03 术语库/`. To rebuild:
|
||||||
1. Convert source xlsx/ods → CSV+YAML in `_output/`
|
1. Convert source xlsx/ods → CSV+YAML in `_output/`
|
||||||
2. Rebuild DuckDB from CSVs
|
2. Load CSVs into SQLite as the `terms` table (zh, en, loc, source)
|
||||||
3. Materialize `unified_terms_flat` view → table for performance
|
|
||||||
|
|
||||||
**Full rebuild pipeline:** See `references/termbase-rebuild.md` (absorbed from the `termbase-management` skill).
|
**Full rebuild pipeline:** See `references/termbase-rebuild.md` (absorbed from the `termbase-management` skill).
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
# Termbase Rebuild (from absorbed termbase-management)
|
# Termbase Rebuild (from absorbed termbase-management)
|
||||||
|
|
||||||
How to rebuild the terms DuckDB from source spreadsheets. This is the full pipeline from the now-archived `termbase-management` skill.
|
How to rebuild the terms SQLite database from source spreadsheets. This is the full pipeline from the now-archived `termbase-management` skill.
|
||||||
|
|
||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install openpyxl odfpy pyyaml duckdb
|
pip install openpyxl odfpy pyyaml
|
||||||
```
|
```
|
||||||
|
|
||||||
## Step 1: Inspect spreadsheet structure
|
## Step 1: Inspect spreadsheet structure
|
||||||
@@ -69,17 +69,23 @@ def dedup_headers(headers):
|
|||||||
- **CSV**: `csv.writer` — column-major, preserves all raw data
|
- **CSV**: `csv.writer` — column-major, preserves all raw data
|
||||||
- **YAML**: `yaml.dump(data, allow_unicode=True, default_flow_style=False, sort_keys=False, width=200)` — list of dicts
|
- **YAML**: `yaml.dump(data, allow_unicode=True, default_flow_style=False, sort_keys=False, width=200)` — list of dicts
|
||||||
|
|
||||||
## Step 3: Load into DuckDB
|
## Step 3: Load into SQLite
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import duckdb
|
import sqlite3
|
||||||
con = duckdb.connect('termlib.duckdb')
|
import csv
|
||||||
|
|
||||||
# Simple CSVs work with auto-detect:
|
con = sqlite3.connect('termlib.sqlite')
|
||||||
con.execute("""
|
|
||||||
CREATE TABLE table_name AS
|
# Simple CSVs work with csv.reader:
|
||||||
SELECT * FROM read_csv_auto('file.csv', header=true, all_varchar=true)
|
with open('file.csv', 'r', encoding='utf-8') as f:
|
||||||
""")
|
rows = list(csv.reader(f))
|
||||||
|
headers = rows[0]
|
||||||
|
data = rows[1:]
|
||||||
|
col_defs = ', '.join(f'"{h}" TEXT' for h in headers)
|
||||||
|
con.execute(f'CREATE TABLE "table_name" ({col_defs})')
|
||||||
|
con.executemany(f'INSERT INTO "table_name" VALUES ({", ".join(["?"] * len(headers))})', data)
|
||||||
|
con.commit()
|
||||||
```
|
```
|
||||||
|
|
||||||
### Pitfall: Multiline CSV fields
|
### Pitfall: Multiline CSV fields
|
||||||
@@ -104,10 +110,10 @@ for i in range(0, len(data), batch_size):
|
|||||||
con.execute(f'INSERT INTO "{table}" VALUES {placeholders}', flat)
|
con.execute(f'INSERT INTO "{table}" VALUES {placeholders}', flat)
|
||||||
```
|
```
|
||||||
|
|
||||||
### Pitfall: DuckDB CLI opens in-memory by default
|
### SQLite CLI
|
||||||
Running plain `duckdb` gives an empty database. Always pass the file path:
|
Open the database file directly:
|
||||||
```
|
```
|
||||||
duckdb path/to/termlib.duckdb
|
sqlite3 path/to/termlib.sqlite
|
||||||
```
|
```
|
||||||
|
|
||||||
## Step 4: Create unified views
|
## Step 4: Create unified views
|
||||||
@@ -122,6 +128,6 @@ See `references/unified-view.sql` for the pattern. Key patterns:
|
|||||||
|
|
||||||
- **ODS reading**: Must traverse `odf.text.P` child elements, not direct text nodes
|
- **ODS reading**: Must traverse `odf.text.P` child elements, not direct text nodes
|
||||||
- **Duplicate headers**: JSON/YAML dict silently overwrites duplicate keys — always deduplicate
|
- **Duplicate headers**: JSON/YAML dict silently overwrites duplicate keys — always deduplicate
|
||||||
- **Multiline CSV + DuckDB**: `read_csv_auto` fails on CSVs with quoted newlines — use Python csv.reader
|
- **Multiline CSV**: Use Python `csv.reader` for CSVs with embedded newlines
|
||||||
- **DuckDB path**: Always explicit file path; `duckdb` alone is in-memory
|
- **SQLite path**: Always pass the file path to `sqlite3`
|
||||||
- **`execute_code` sandbox**: Does NOT share pip-installed packages — use `terminal` for Python scripts
|
- **`execute_code` sandbox**: Does NOT share pip-installed packages — use `terminal` for Python scripts
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ After producing a first-pass translation, or when the user asks to check termino
|
|||||||
|
|
||||||
1. Read the full translated file. Extract all Chinese terms from `{% "TERM" (pinyin) = ENGLISH ... %}` blocks.
|
1. Read the full translated file. Extract all Chinese terms from `{% "TERM" (pinyin) = ENGLISH ... %}` blocks.
|
||||||
|
|
||||||
2. Start the search server: `python3 $MPI_PROJECT_ROOT/toolkit/terms-database/server.py &` (port 8910). It may already be running — check with `curl -s http://localhost:8910/`.
|
2. Start the search server: `python3 toolkit/terms-database/server.py &` (port 8910). It may already be running — check with `curl -s http://localhost:8910/`.
|
||||||
|
|
||||||
3. Batch-search each term via the HTTP API:
|
3. Batch-search each term via the HTTP API:
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -1,13 +1,9 @@
|
|||||||
---
|
---
|
||||||
name: translation-review
|
name: mpi-translation-review
|
||||||
description: |
|
description: |
|
||||||
Unified review skill for Chinese-English Buddhist/Dharma translations.
|
Unified review skill for Chinese-English Buddhist/Dharma translations in djot format.
|
||||||
Supports two modes:
|
Use in self mode to edit your own target.dj, or in other mode to write review-comments.dj for a peer translator.
|
||||||
- **self**: You translated the text. Edit target.dj directly.
|
Do not use for non-djot formats or for non-Buddhist texts.
|
||||||
- **other**: Someone else translated. Write review-comments.dj, do not edit target.dj.
|
|
||||||
What to review is identical in both modes — the same detection rules,
|
|
||||||
editorial standards, and terminology checks. Only the action differs: self
|
|
||||||
mode applies fixes directly; other mode records them for the translator.
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# Translation Review (unified)
|
# Translation Review (unified)
|
||||||
@@ -34,12 +30,17 @@ Switch to self-mode output in that case, but keep a collegial tone.
|
|||||||
|
|
||||||
1. Read `source.dj` and `target.dj` fully before changing anything.
|
1. Read `source.dj` and `target.dj` fully before changing anything.
|
||||||
(Other mode only: read the translator's note if present, and address them by name.)
|
(Other mode only: read the translator's note if present, and address them by name.)
|
||||||
2. Run a three-pass review using the detection rules below.
|
2. If the text is **meditation-practice content** — a guided meditation script,
|
||||||
3. Apply the R1–R14 editorial polish checklist.
|
posture/breathing exercise guide, or meditation-method explanation — load
|
||||||
4. Check terminology against the terms DB (see `mpi-terms-search` skill).
|
`references/meditation-practice-translation.md` and run its summary checklist
|
||||||
5. Verify that every source paragraph maps to a target paragraph with no missing
|
as an extra pass. It distills the recurring issues a human reviewer flagged
|
||||||
|
across a whole meditation manuscript.
|
||||||
|
3. Run a three-pass review using the detection rules below.
|
||||||
|
4. Apply the R1–R14 editorial polish checklist.
|
||||||
|
5. Check terminology against the terms DB (see `mpi-terms-search` skill).
|
||||||
|
6. Verify that every source paragraph maps to a target paragraph with no missing
|
||||||
or truncated content.
|
or truncated content.
|
||||||
6. Produce the correct artifact for your mode.
|
7. Produce the correct artifact for your mode.
|
||||||
|
|
||||||
## Detection rules — three passes
|
## Detection rules — three passes
|
||||||
|
|
||||||
@@ -89,9 +90,13 @@ issue with line/paragraph references.
|
|||||||
- **D2 Cultural anachronism.** Modern concepts projected onto classical material.
|
- **D2 Cultural anachronism.** Modern concepts projected onto classical material.
|
||||||
Example: translating 般若 as "wisdom" in a scholarly context may flatten the
|
Example: translating 般若 as "wisdom" in a scholarly context may flatten the
|
||||||
term; in a popular talk it may be exactly right.
|
term; in a popular talk it may be exactly right.
|
||||||
- **D3 Tone / voice of the teacher.** For oral talks, preserve the speaker's
|
- **D3 Tone AND voice (语气) of the teacher.** For oral talks, preserve the
|
||||||
warmth, rhetorical questions, and direct address. Do not flatten into essay
|
speaker's warmth, rhetorical questions, and direct address. Beyond register,
|
||||||
prose.
|
check the 语气 markers in `../mpi-translation/SKILL.md` → "Speaker's Voice
|
||||||
|
(语气)": rhetorical questions kept as questions, first-person teacher asides
|
||||||
|
("我经常说") kept in first person, reasoning connectives (可见, 所以说)
|
||||||
|
preserved, inclusive we/you address, homely analogies left concrete, gentle
|
||||||
|
rather than scolding admonition. Do not flatten into essay prose.
|
||||||
- **D4 Implicit meaning / implicature.** What the source implies but does not say
|
- **D4 Implicit meaning / implicature.** What the source implies but does not say
|
||||||
(e.g., irony, conventional politeness, Gricean maxims). Ensure the implication
|
(e.g., irony, conventional politeness, Gricean maxims). Ensure the implication
|
||||||
survives or is compensated.
|
survives or is compensated.
|
||||||
@@ -207,6 +212,10 @@ Any DB queries or proposed term changes.
|
|||||||
for this project's genres.
|
for this project's genres.
|
||||||
- `references/buddhist-terminology.md` — register and convention notes for
|
- `references/buddhist-terminology.md` — register and convention notes for
|
||||||
Buddhist/Dharma terms.
|
Buddhist/Dharma terms.
|
||||||
|
- `references/meditation-practice-translation.md` — recurring review patterns
|
||||||
|
for meditation-practice texts (person/subject consistency, sentence
|
||||||
|
splitting, filler deletion, fixed series terms, Chan-verse quotes), distilled
|
||||||
|
from a human-reviewed manuscript.
|
||||||
- `../mpi-translation/SKILL.md` — the upstream translation skill that produces the
|
- `../mpi-translation/SKILL.md` — the upstream translation skill that produces the
|
||||||
`target.dj` this skill reviews.
|
`target.dj` this skill reviews.
|
||||||
- `../mpi-terms-search/SKILL.md` — skill for querying the terms database before and
|
- `../mpi-terms-search/SKILL.md` — skill for querying the terms database before and
|
||||||
|
|||||||
@@ -35,6 +35,29 @@ Terms encountered in Chinese-English translation of Dharma study materials. Thes
|
|||||||
| 愿心 | mind of vows, bodhicitta aspiration | Plural "vows" in English |
|
| 愿心 | mind of vows, bodhicitta aspiration | Plural "vows" in English |
|
||||||
| 重要感、优越感、主宰欲 | sense of importance, superiority, desire to control | Three ego-driven motivations |
|
| 重要感、优越感、主宰欲 | sense of importance, superiority, desire to control | Three ego-driven motivations |
|
||||||
|
|
||||||
|
## Meditation-practice terms (from 初级正念禅修 review)
|
||||||
|
|
||||||
|
Terms from the *初级正念禅修* review (2026-08-10). These appear in
|
||||||
|
meditation-method / breathing / posture texts. See
|
||||||
|
`references/meditation-practice-translation.md` for the full review patterns.
|
||||||
|
|
||||||
|
| Chinese | English | Notes |
|
||||||
|
|---------|---------|-------|
|
||||||
|
| 初级正念(禅修) | Primary Mindfulness (Meditation) | NOT "elementary". 初级 → "primary" in this series. |
|
||||||
|
| 毗卢七支坐 | seven-point posture of Vairocana | terms DB: "Seven-Point Postures of Vairochana". Use consistently. |
|
||||||
|
| 八式动禅 | Eight Mindful Exercises for Lotus Posture | Fixed series term. |
|
||||||
|
| 七式呼吸 | Seven Mindful Exercises for Breathing | Fixed series term. |
|
||||||
|
| 观呼吸 | Mindful Breathing | As a practice name; not "Mindfulness of Breathing". |
|
||||||
|
| 打坐 / 坐禅 | sitting meditation | terms DB `坐禅`. Use uniformly ("seated meditation" OK). |
|
||||||
|
| 细(呼吸) | subtle | For 细 breath quality; "fine" reads unnatural to general readers. |
|
||||||
|
| 止 | settling the mind on an object | Terse source needs expansion; never bare "stopping". |
|
||||||
|
| 证悟 | enlightenment | terms DB. Not "Way". |
|
||||||
|
| 本心 / 真心 | true mind | Use consistently within a document (also for 心性 where intended). |
|
||||||
|
| 辅助(禅修) | supportive (mindfulness practice) | "supportive" preferred over "auxiliary" (warmer, conversational). |
|
||||||
|
| 觉知 | awareness | terms DB. |
|
||||||
|
| 观照 | attend to the mind / mindful observation | Contemplative, not passive "observe". |
|
||||||
|
| 缘起 | dependent origination | Prefer "follow the principle of dependent origination" collocation. |
|
||||||
|
|
||||||
## Structural patterns
|
## Structural patterns
|
||||||
|
|
||||||
- Section numbering: Chinese uses 一、二、三... English should pick one style (Part One/Two, First/Second, I/II) and stick with it.
|
- Section numbering: Chinese uses 一、二、三... English should pick one style (Part One/Two, First/Second, I/II) and stick with it.
|
||||||
|
|||||||
@@ -0,0 +1,174 @@
|
|||||||
|
# Meditation-Practice Text Translation — Patterns from Human Review
|
||||||
|
|
||||||
|
Human review of *初级正念禅修:从基础练习到觉知开启* (translated 代艺初翻,
|
||||||
|
reviewed 慈德审, 2026-08-10) produced 85 reviewer comments. They cluster into a
|
||||||
|
small set of recurring patterns. These are the issues a Chinese → English
|
||||||
|
translator/reviewer of meditation-practice texts most often gets wrong. Treat
|
||||||
|
this as a checklist for any guided-meditation script, posture/breathing
|
||||||
|
exercise guide, or meditation-method explanation.
|
||||||
|
|
||||||
|
## 1. Person and subject (the single most-repeated issue)
|
||||||
|
|
||||||
|
Chinese meditation prose drops subjects; English must not. The reviewer's #1
|
||||||
|
fix was to **supply and keep a consistent person as subject** — almost always
|
||||||
|
inclusive **"we"**, with **"you"** for direct step-by-step guidance. Two
|
||||||
|
failures dominate:
|
||||||
|
|
||||||
|
- **Missing subject / missing verb.** A bare clause that reads as an imperative
|
||||||
|
fragment where the source intended a statement.
|
||||||
|
- Source: `……入静` → target `, adjusting ...` → fix: give it a subject and a
|
||||||
|
full verb.
|
||||||
|
- **Drift into abstract third person / generic "people"/"one".** Meditation
|
||||||
|
practice is reader-facing. Render the practitioner as "we" (shared
|
||||||
|
experience) or "you" (guidance), never as "people," "one," or a dangling
|
||||||
|
noun phrase.
|
||||||
|
|
||||||
|
**Rule:** For every sentence, name the subject. Check the previous sentence's
|
||||||
|
subject and keep the reference chain consistent (don't hop between "the
|
||||||
|
practitioner," "we," "you," and an inanimate object). The reviewer repeatedly
|
||||||
|
asked to change the subject *back* to a consistent person for reader
|
||||||
|
engagement (参与感).
|
||||||
|
|
||||||
|
Examples from the review:
|
||||||
|
|
||||||
|
| Problem | Reviewed fix |
|
||||||
|
|---|---|
|
||||||
|
| `use` (no subject) | "In the Seven Mindful Exercises for Breathing, **we use** postures and various body movements to expand the space of breathing…" |
|
||||||
|
| "process" as subject | "**In this process, we need to keep alert**: when we realize that the mind has wandered off or grown drowsy, we bring it back." |
|
||||||
|
| missing subject at paragraph start | supply "we": "Through repeated practice, **we** gradually master it and make it a habit." |
|
||||||
|
| "when there is no awareness…" | "**When we** are not aware…" |
|
||||||
|
|
||||||
|
When the source uses an abstract/system noun (mind, meditation, the method),
|
||||||
|
match it only if the English sentence genuinely reads well that way; if it
|
||||||
|
reads stiff, re-cast around a person. Chinese topic-comment structure must not
|
||||||
|
be reproduced as a subjectless English sentence.
|
||||||
|
|
||||||
|
## 2. Split long sentences (one idea per breath)
|
||||||
|
|
||||||
|
Chinese runs clauses together; the reviewer repeatedly split one long sentence
|
||||||
|
into two or three, often introducing "we" as the subject of each.
|
||||||
|
|
||||||
|
| Before (literal) | After (reviewed) |
|
||||||
|
|---|---|
|
||||||
|
| "We need the view of the sudden teaching so that we can be filled with confidence in practice—knowing that within the life of every person lies a quality no different from that of all the Buddhas of the three periods of time and the successive patriarchs" | "We need the view of the sudden teaching because it gives us confidence in our practice. It helps us understand that within each of us lies a quality no different from that of all the Buddhas across the past, present, and future, as well as the successive patriarchs." |
|
||||||
|
| "These exercises rely not only on movement practice but also on coordinating with the breath—during each movement training, bringing the breath to every part of the body…" | "In these exercises, we not only rely on physical movement but also regulate the breath. During each movement, we bring the breath to every part of the body…" |
|
||||||
|
| "As long as we perform each movement with concentration and awareness, then whether we are dressing, eating, relating with others, walking…" | "As long as we engage in each action with concentration and awareness, even daily activities can become the actual practice of meditation—whether we are getting dressed, eating, walking, standing, sitting, lying down, interacting with others, or handling daily affairs." |
|
||||||
|
|
||||||
|
## 3. Delete redundant filler (the second most-repeated fix)
|
||||||
|
|
||||||
|
The reviewer deleted words that add nothing: `particular` (with `emphasis`),
|
||||||
|
`corresponding`, `especially`, `basically`, `essentially`, `like someone`,
|
||||||
|
`in fantasy`, `the`, `a process of`, `arising and passing`, `, the crucial
|
||||||
|
doorway to awakened nature`, `to heed during sitting meditation`.
|
||||||
|
|
||||||
|
**Rule:** When a sentence parses with and without a word, drop it. Common
|
||||||
|
Chinese-draft padding: "especially," "basically," "essentially,"
|
||||||
|
"particularly," "corresponding," "a process of." These are almost always
|
||||||
|
compressible or deletable.
|
||||||
|
|
||||||
|
## 4. Over-literal / word-for-word (字对字) renderings
|
||||||
|
|
||||||
|
The reviewer flagged machine-literal translations that needed to become
|
||||||
|
natural, person-led English:
|
||||||
|
|
||||||
|
| Over-literal | Reviewed fix |
|
||||||
|
|---|---|
|
||||||
|
| "life-form" (for 色身/生命载体) | "Our present body, made up of the five aggregates, is…" |
|
||||||
|
| "even dull faculties, provided the method is right…" | "Even those of dull faculties, if they practice with the right method and diligently wipe away the dust of the mind, can realize the true mind." |
|
||||||
|
| "when able, do abdominal breathing" (字对字 of 能则) | "When we are able to practice abdominal breathing, we simply do it." |
|
||||||
|
| "we climb on again" (for a car returning to a highway) | "we get back on" — the AI misread the metaphor; a car does not "climb." |
|
||||||
|
| "we are accustomed to 'not knowing'" | "we are used to being unaware" |
|
||||||
|
|
||||||
|
**Rule:** If the English only makes sense by re-translating it into the
|
||||||
|
Chinese, it is over-literal. Read for meaning, not word order.
|
||||||
|
|
||||||
|
## 5. Avoid exaggerated or unusual English
|
||||||
|
|
||||||
|
Chinese meditation prose is plain; keep the English plain. The reviewer
|
||||||
|
specifically softened:
|
||||||
|
|
||||||
|
- "an unending supply of vitality and energy" → "the vitality and energy
|
||||||
|
needed" / "a steady supply" (unending sounds exaggerated).
|
||||||
|
- "like a turbid pool slowly becoming clear" → "like a pool of muddy water
|
||||||
|
slowly becoming clear" (turbid is literary; muddy is plain).
|
||||||
|
- "fine" (for 细 breath) → "subtle" — meditative, natural for general readers.
|
||||||
|
- "deconstruct" → "see through" (plainer).
|
||||||
|
- "obscured" → "veiled" (fewer, simpler words).
|
||||||
|
- "transcend" → "go beyond" (especially when another "transcend" appears nearby
|
||||||
|
— avoid repetition).
|
||||||
|
- "accord with" → "align with" (more conversational for a general audience).
|
||||||
|
|
||||||
|
**Rule:** Prefer short, warm, everyday words over literary or Latinate ones
|
||||||
|
for a lay readership. If a word would need a dictionary look-up or a footnote
|
||||||
|
for a general reader, replace it.
|
||||||
|
|
||||||
|
## 6. Consistency of terms within the document
|
||||||
|
|
||||||
|
- Same concept, one rendering: 本心/真心 → **true mind** throughout (not
|
||||||
|
alternating "true mind" / "nature" / "original mind"). The reviewer picked
|
||||||
|
"true mind" as the clearest and asked for it to be used consistently.
|
||||||
|
- 初级 → **primary** (not "elementary"). 初级正念禅修 = **Primary Mindfulness
|
||||||
|
Meditation**. Pick one form for "elementary/primary/basic" and keep it.
|
||||||
|
- 打坐/坐禅 → **sitting meditation** (uniform; matches terms DB `坐禅` →
|
||||||
|
"sitting meditation").
|
||||||
|
- Section-title style: 观呼吸 → **Mindful Breathing** (not "Mindfulness of
|
||||||
|
Breathing") when used as a practice name.
|
||||||
|
|
||||||
|
**Rule:** Build a one-term-per-concept map for the file and enforce it. Any
|
||||||
|
term changed mid-review must be updated everywhere (search the whole file).
|
||||||
|
|
||||||
|
## 7. Fixed series / internal terms (check the series, not just the DB)
|
||||||
|
|
||||||
|
This book belongs to a meditation series with established names. Do not invent
|
||||||
|
new renderings when a fixed name exists:
|
||||||
|
|
||||||
|
- 毗卢七支坐 → **seven-point posture of Vairocana** (terms DB: "Seven-Point
|
||||||
|
Postures of Vairochana").
|
||||||
|
- 八式动禅 → **Eight Mindful Exercises for Lotus Posture**.
|
||||||
|
- 七式呼吸 → **Seven Mindful Exercises for Breathing**.
|
||||||
|
- 辅助 (as in auxiliary mindfulness practice) → **supportive** (reviewer
|
||||||
|
preferred "supportive" over "auxiliary": warmer, more conversational).
|
||||||
|
|
||||||
|
**Rule:** For a multi-part practice system or series, check whether the names
|
||||||
|
are already established in the series / terms DB before coining new English.
|
||||||
|
|
||||||
|
## 8. Terse Dharma statements need expansion, not word-for-word
|
||||||
|
|
||||||
|
A terse Chinese phrase ("止就是安住", "只是转换一下") is often too compressed
|
||||||
|
to render literally. Expand it into a complete, teachable English sentence:
|
||||||
|
|
||||||
|
| Terse | Reviewed expansion |
|
||||||
|
|---|---|
|
||||||
|
| "stopping" means settling | "We need to rest the mind on the object of focus; *samatha* means settling the mind on an object." |
|
||||||
|
| "originally not a single thing" / "at all times diligently wipe it clean" (止观 contrast) | "Sudden teachings point out that the true mind is originally free from all attachment, while gradual teachings emphasize that we, as ordinary beings, need to diligently wipe away the inner dust." |
|
||||||
|
| "…anxious about…" → impermanence point | "Impermanence: It helps us understand that thoughts arise and cease. With awareness, we can observe them coming and going without seeing them as permanent." |
|
||||||
|
|
||||||
|
## 9. Chan verse / scripture quotes — reuse established renderings
|
||||||
|
|
||||||
|
When the text quotes a famous Chan verse, do NOT translate it fresh. Use the
|
||||||
|
established English rendering from the lineage, and if the project already
|
||||||
|
published this verse (an earlier book in the series), reuse that exact
|
||||||
|
wording.
|
||||||
|
|
||||||
|
Example (信心铭 / *Faith in Mind*, 三祖僧璨): the source quotes
|
||||||
|
"至道无难,唯嫌拣择,但莫憎爱,洞然明白". The reviewer noted the English
|
||||||
|
target mis-quoted the verse and that the correct rendering ("Without love or
|
||||||
|
hate, one sees things as they truly are") was already used in an earlier
|
||||||
|
translated book in the series. Check `references/translation-pitfalls.md` →
|
||||||
|
"Chan verse / scripture quotes" and prior published books before translating.
|
||||||
|
|
||||||
|
## Summary checklist for meditation-practice texts
|
||||||
|
|
||||||
|
- [ ] Every sentence has an explicit, consistent person subject ("we"/"you"),
|
||||||
|
no abstract-third-person drift, no subjectless fragments.
|
||||||
|
- [ ] Long sentences split into breath-length units.
|
||||||
|
- [ ] No redundant filler ("especially," "basically," "essentially,"
|
||||||
|
"particularly," "corresponding").
|
||||||
|
- [ ] No word-for-word (字对字) renderings; English reads for meaning.
|
||||||
|
- [ ] No exaggerated/literary words; warm, plain vocabulary.
|
||||||
|
- [ ] One term per concept, enforced file-wide (true mind, primary, sitting
|
||||||
|
meditation, Mindful Breathing).
|
||||||
|
- [ ] Fixed series terms (Vairocana posture, Eight Mindful Exercises for Lotus
|
||||||
|
Posture, Seven Mindful Exercises for Breathing, supportive).
|
||||||
|
- [ ] Terse Dharma statements expanded into teachable sentences.
|
||||||
|
- [ ] Chan verse quotes use established published renderings, not fresh drafts.
|
||||||
@@ -17,9 +17,12 @@ correspondence, generate `bilingual.dj` directly from the DOCX:
|
|||||||
4. Write bilingual.dj
|
4. Write bilingual.dj
|
||||||
The DOCX English is the authoritative target text. No PDF needed.
|
The DOCX English is the authoritative target text. No PDF needed.
|
||||||
|
|
||||||
**Extraction approach**: start by adapting `toolkit/scripts/gen-bilingual-docx.py`.
|
**Extraction approach**: write a custom extraction script. The standard
|
||||||
For articles where the body has strict CN→EN→CN→EN alternation, the simple
|
`toolkit/scripts/gen-bilingual.py` expects separate `source.dj` and `target.dj` files;
|
||||||
extraction in that script (CN line, blank, EN line, blank) works directly.
|
for DOCX→bilingual extraction, adapt its pattern-matching logic to read from the
|
||||||
|
pandoc plain-text output instead. For articles where the body has strict
|
||||||
|
CN→EN→CN→EN alternation, the simple extraction (CN line, blank, EN line, blank)
|
||||||
|
works directly.
|
||||||
|
|
||||||
### A2. Bilingual from `.docx.md` (pandoc markdown output)
|
### A2. Bilingual from `.docx.md` (pandoc markdown output)
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ Import directly in `execute_code` scripts — no subprocess, no server, no text
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
import sys
|
import sys
|
||||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
sys.path.insert(0, 'toolkit/terms-database')
|
||||||
from search import search
|
from search import search
|
||||||
|
|
||||||
results = search("三级修学", limit=5)
|
results = search("三级修学", limit=5)
|
||||||
@@ -20,7 +20,7 @@ results = search("空性", loc="心经", src="DoT定稿", limit=5)
|
|||||||
|
|
||||||
```python
|
```python
|
||||||
import sys
|
import sys
|
||||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
sys.path.insert(0, 'toolkit/terms-database')
|
||||||
from search import search
|
from search import search
|
||||||
|
|
||||||
terms = ["三无漏学", "八步三禅", "闻思修", ...]
|
terms = ["三无漏学", "八步三禅", "闻思修", ...]
|
||||||
|
|||||||
@@ -275,6 +275,37 @@ initial/middle/great scope."
|
|||||||
|
|
||||||
佛教术语 has "Guanshiyin/Guanyin Bodhisattva." Use "Guanyin Bodhisattva."
|
佛教术语 has "Guanshiyin/Guanyin Bodhisattva." Use "Guanyin Bodhisattva."
|
||||||
|
|
||||||
|
## Chan verse / scripture quotes — reuse established renderings
|
||||||
|
|
||||||
|
When the source quotes a well-known Chan verse or sutra passage, do NOT draft
|
||||||
|
a fresh translation. Use the established English rendering from the lineage,
|
||||||
|
and check whether the project already published this verse in an earlier book
|
||||||
|
of the series — reuse that exact wording.
|
||||||
|
|
||||||
|
**Case (信心铭 / *Faith in Mind*, from 初级正念禅修 review):** the source
|
||||||
|
quoted "至道无难,唯嫌拣择,但莫憎爱,洞然明白". The draft target mis-quoted
|
||||||
|
the verse (anchoring the wrong lines) and the reviewer noted the correct
|
||||||
|
rendering ("Without love or hate, one sees things as they truly are.") was
|
||||||
|
already used in an earlier translated book in the series. The annotated
|
||||||
|
"才有是非,纷然失心" was a mis-annotation of the source; the passage the
|
||||||
|
source actually quotes is the four-line verse above.
|
||||||
|
|
||||||
|
**Detection:** For any quoted verse/idiom the source sets off (quotation
|
||||||
|
marks, 曰/云/云何 structures), identify the source work (e.g. 信心铭, 金刚经,
|
||||||
|
六祖坛经) and search prior published translations in the series before
|
||||||
|
translating. A famous Chan line almost always has a standard English form.
|
||||||
|
|
||||||
|
## AI misinterpretation of Chinese metaphor
|
||||||
|
|
||||||
|
LLMs sometimes mistranslate a concrete Chinese metaphor by guessing the wrong
|
||||||
|
image. When the English rendering of a metaphor is physically impossible or
|
||||||
|
odd ("we climb on again" for a car returning to a highway), the model
|
||||||
|
misread the source, not the register. Re-read the source image literally and
|
||||||
|
render the actual picture in plain English ("we get back on").
|
||||||
|
|
||||||
|
**Rule:** If the English metaphor is hard to visualize or sounds comical, it
|
||||||
|
is likely a mistranslation of the source image — check the Chinese.
|
||||||
|
|
||||||
## Workflow pitfall
|
## Workflow pitfall
|
||||||
|
|
||||||
### Translating before consulting terms DB
|
### Translating before consulting terms DB
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
---
|
---
|
||||||
name: translation
|
name: mpi-translation
|
||||||
description: Translate Chinese↔English Buddhist/Dharma content — register guidance from Mindfulness Bell corpus, tone, voice, cultural bridging technique.
|
description: Translate Chinese↔English Buddhist/Dharma content. Use for oral talks, written articles, guided meditations, Q&A, and sutra commentary. Do not use for non-religious general Chinese-English translation, technical documentation, or marketing copy.
|
||||||
inputs: source.dj (Chinese djot), or .docx via docx2dj.fish
|
inputs: source.dj (Chinese djot), or .docx via docx2dj.fish
|
||||||
outputs: target.dj (English djot), bilingual.dj, edit-suggestions.dj
|
outputs: target.dj (English djot), bilingual.dj, edit-suggestions.dj
|
||||||
|
compatibility: Requires pandoc for docx-to-djot conversion.
|
||||||
---
|
---
|
||||||
|
|
||||||
# Translation
|
# Translation
|
||||||
@@ -12,7 +13,7 @@ terms DB query, workflows, output format) are in AGENTS.md.
|
|||||||
|
|
||||||
## Source context
|
## Source context
|
||||||
|
|
||||||
Before translating, understand the source's format and delivery context. Is it a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, a written article, or another genre? The register shapes the translation. If the context is not clear from the file path or source content, ask the user before proceeding.
|
Before translating, understand the source's format and delivery context. Is it a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, a written article, or another genre? The register shapes the translation. If the context is not clear from the file path or source content, ask the user before proceeding. If you cannot identify the author, write the target text with the style of 济群法师.
|
||||||
|
|
||||||
## Mindfulness Bell Corpus
|
## Mindfulness Bell Corpus
|
||||||
|
|
||||||
@@ -40,6 +41,40 @@ Quick-find in index: Thầy talks → `"Thích Nhất Hạnh"` + page ≤ 10; te
|
|||||||
4. **Voice**: Direct address ("you"), concrete images, and oral rhythm make Dharma land in English. Abstract noun chains (common in Chinese→English translationese) kill it.
|
4. **Voice**: Direct address ("you"), concrete images, and oral rhythm make Dharma land in English. Abstract noun chains (common in Chinese→English translationese) kill it.
|
||||||
5. **Sutra quotes**: Use standard English Buddhist idiom. Check terse-idiom conventions (e.g., Diamond Sutra "lives" not "bodies").
|
5. **Sutra quotes**: Use standard English Buddhist idiom. Check terse-idiom conventions (e.g., Diamond Sutra "lives" not "bodies").
|
||||||
|
|
||||||
|
## Speaker's Voice (语气)
|
||||||
|
|
||||||
|
Tone (register, formality) is not enough — preserve the speaker's 语气: the
|
||||||
|
stance and manner carried by sentence mood. Reviewers have flagged translations
|
||||||
|
that got the tone right but flattened the teacher's voice. 语气 lives in:
|
||||||
|
|
||||||
|
- **Sentence mood.** Rhetorical questions stay questions ("幸福在哪里?" →
|
||||||
|
"Where is happiness?", not "Happiness is nowhere to be found.").
|
||||||
|
Exclamations and wonder stay exclamatory. Do not convert the speaker's
|
||||||
|
questioning into declarations.
|
||||||
|
- **First-person teacher asides.** 济群法师 often speaks in his own voice:
|
||||||
|
"我经常说……", "我曾在讲座中多次谈到……", "由此我想到……". Keep the
|
||||||
|
first person — do not flatten to "it is often said" or impersonal prose.
|
||||||
|
These asides are how he establishes presence with the audience.
|
||||||
|
- **Reasoning connectives.** His talks argue step by step: 可见, 所以说,
|
||||||
|
事实上, 问题在于. Render the logical gait ("So it is clear that…", "That
|
||||||
|
is why…", "In fact…") rather than dropping it — the reasoned,
|
||||||
|
unhurried persuasion IS his voice.
|
||||||
|
- **Inclusive address.** Default "we" for shared human condition, "you" when
|
||||||
|
he turns to the listener. Do not drift into abstract third person ("one",
|
||||||
|
"people") where the source speaks as teacher-to-audience.
|
||||||
|
- **Everyday analogies, plainly told.** Rotting apples, leaking boats,
|
||||||
|
teacups, face masks — keep the homely image concrete and unvarnished;
|
||||||
|
do not upgrade it to literary language or explain it away.
|
||||||
|
- **Gentle admonition, never scolding.** He points out folly with warmth and
|
||||||
|
a little humor (the "有点烦" pop song, Mo Yan's dodge). Keep the lightness;
|
||||||
|
avoid both sermonizing severity and jokey casualness.
|
||||||
|
- **Measured authority.** Calm, composed, unhurried. No hype, no exclamation-
|
||||||
|
point enthusiasm, no academic hedging ("arguably", "it could be said").
|
||||||
|
|
||||||
|
Check: read a paragraph aloud as if delivering a talk to a lay audience. If it
|
||||||
|
sounds like an essay, a lecture abstract, or a motivational speaker, the 语气
|
||||||
|
has been lost.
|
||||||
|
|
||||||
## Pitfalls
|
## Pitfalls
|
||||||
|
|
||||||
### Pre-flight accuracy check
|
### Pre-flight accuracy check
|
||||||
@@ -241,6 +276,10 @@ Read the entire English target aloud. If anything stalls, rephrase it.
|
|||||||
|
|
||||||
When translating guided meditation scripts, exercise guides, or posture instructions
|
When translating guided meditation scripts, exercise guides, or posture instructions
|
||||||
(rather than Dharma talks), use a lighter workflow. See `references/meditation-translation.md`.
|
(rather than Dharma talks), use a lighter workflow. See `references/meditation-translation.md`.
|
||||||
|
Before drafting, also read `../mpi-translation-review/references/meditation-practice-translation.md`
|
||||||
|
(patterns a human reviewer flagged across a meditation manuscript) so the recurring
|
||||||
|
issues — dropped/inconsistent person, over-long sentences, filler, 字对字
|
||||||
|
renderings, fixed series terms — are avoided up front rather than caught at review.
|
||||||
|
|
||||||
## Other Pitfalls
|
## Other Pitfalls
|
||||||
|
|
||||||
@@ -253,6 +292,7 @@ Create article-specific scripts per `references/proofread-pdf-workflow.md`.
|
|||||||
## References
|
## References
|
||||||
|
|
||||||
- `references/meditation-translation.md` — lighter workflow for meditation/mindfulness content
|
- `references/meditation-translation.md` — lighter workflow for meditation/mindfulness content
|
||||||
|
- `../mpi-translation-review/references/meditation-practice-translation.md` (cross-skill) — recurring patterns for meditation-practice texts, distilled from a human-reviewed manuscript
|
||||||
- `references/markdown-to-djot.md` — converting .docx.md to .dj for translation prep
|
- `references/markdown-to-djot.md` — converting .docx.md to .dj for translation prep
|
||||||
- `references/bilingual-format.md` — bilingual.dj layout: source/target adjacent, blank separator between pairs
|
- `references/bilingual-format.md` — bilingual.dj layout: source/target adjacent, blank separator between pairs
|
||||||
- `references/diacritics-convention.md` — diacritics rules
|
- `references/diacritics-convention.md` — diacritics rules
|
||||||
|
|||||||
@@ -7,7 +7,8 @@ use a lighter workflow than the full dharma-translation pipeline.
|
|||||||
## Register
|
## Register
|
||||||
|
|
||||||
Default to warm, direct instructional voice (Thầy-adjacent):
|
Default to warm, direct instructional voice (Thầy-adjacent):
|
||||||
- Second-person address ("you")
|
- Inclusive first person ("we") for describing the practice and shared
|
||||||
|
experience; second person ("you") for direct step-by-step guidance
|
||||||
- Concrete images, sensory details
|
- Concrete images, sensory details
|
||||||
- Oral rhythm, short sentences
|
- Oral rhythm, short sentences
|
||||||
- Present tense, imperative mood
|
- Present tense, imperative mood
|
||||||
@@ -15,6 +16,18 @@ Default to warm, direct instructional voice (Thầy-adjacent):
|
|||||||
MB corpus consultation is NOT needed for register — this content type has its own
|
MB corpus consultation is NOT needed for register — this content type has its own
|
||||||
well-established English conventions (yoga/meditation instructional voice).
|
well-established English conventions (yoga/meditation instructional voice).
|
||||||
|
|
||||||
|
### Person and subject (most-reviewed issue in this genre)
|
||||||
|
|
||||||
|
Chinese meditation prose drops subjects; English must not. The human reviewer's
|
||||||
|
#1 fix was to supply and keep a **consistent person as subject** — almost
|
||||||
|
always "we" for shared experience, "you" for direct guidance — and never to
|
||||||
|
drift into abstract third person ("people," "one") or leave a sentence
|
||||||
|
subjectless. Every sentence needs a named subject, and the reference chain
|
||||||
|
across sentences must stay consistent (don't hop between "we," "the
|
||||||
|
practitioner," and an inanimate noun). For the full pattern library from a
|
||||||
|
human-reviewed manuscript, read
|
||||||
|
`../mpi-translation-review/references/meditation-practice-translation.md`.
|
||||||
|
|
||||||
## Terms
|
## Terms
|
||||||
|
|
||||||
Terms DB lookup for Buddhist-mindfulness vocabulary is useful but limited to key terms:
|
Terms DB lookup for Buddhist-mindfulness vocabulary is useful but limited to key terms:
|
||||||
|
|||||||
+21
-5
@@ -1,18 +1,34 @@
|
|||||||
# MPI Skills
|
# MPI Skills
|
||||||
|
|
||||||
These skills live in this directory and are loaded via `~/.hermes/config.yaml`.
|
These skills live in this directory and are loaded by your agent from this path.
|
||||||
|
|
||||||
## Install for Hermes
|
## Install
|
||||||
|
|
||||||
Add to `~/.hermes/config.yaml`:
|
Add the absolute path to this directory to your agent's external skill paths.
|
||||||
|
For Oh My Pi and compatible agents, this is usually a config file with an
|
||||||
|
`external_dirs` list:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
skills:
|
skills:
|
||||||
external_dirs:
|
external_dirs:
|
||||||
- $MPI_PROJECT_ROOT/toolkit/skills
|
- /path/to/mpi/toolkit/skills
|
||||||
```
|
```
|
||||||
|
|
||||||
{% Edit config.yaml directly — `hermes config set` stores list values as strings. %}
|
{% Replace `/path/to/mpi` with the absolute path to your workspace. %}
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
Each skill follows the Agent Skills layout:
|
||||||
|
|
||||||
|
```
|
||||||
|
skill-name/
|
||||||
|
├── SKILL.md # frontmatter + core instructions
|
||||||
|
├── scripts/ # tiny deterministic CLIs
|
||||||
|
├── references/ # docs loaded on demand
|
||||||
|
└── assets/ # templates and static files
|
||||||
|
```
|
||||||
|
|
||||||
|
The `name` field in each `SKILL.md` frontmatter matches the directory name.
|
||||||
|
|
||||||
## Skills
|
## Skills
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ FROM python:3.14-slim
|
|||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
RUN pip install --no-cache-dir flask duckdb gunicorn
|
RUN pip install --no-cache-dir flask gunicorn
|
||||||
|
|
||||||
COPY . .
|
COPY . .
|
||||||
|
|
||||||
|
|||||||
+16
-11
@@ -1,26 +1,31 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env -S uv run --script
|
||||||
"""Full-text search over MPI term database. Queries unified_terms_flat via DuckDB LIKE.
|
# /// script
|
||||||
|
# dependencies = []
|
||||||
|
# ///
|
||||||
|
"""Full-text search over MPI term database (SQLite).
|
||||||
|
|
||||||
Module usage:
|
Module usage:
|
||||||
from search import search_terms
|
from search import search
|
||||||
results = search_terms("空性")
|
results = search("空性")
|
||||||
results = search_terms("空性", limit=5, loc="心经", src="佛教术语")
|
results = search("空性", limit=5, loc="心经", src="佛教术语")
|
||||||
# returns list of dicts: {zh, en, loc, source}
|
# returns list of dicts: {zh, en, loc, source}
|
||||||
|
|
||||||
CLI usage:
|
CLI usage:
|
||||||
python search.py <query> [limit]
|
toolkit/terms-database/search.py <query> [limit]
|
||||||
python search.py 空性 loc:心经 src:公案
|
toolkit/terms-database/search.py 空性 loc:心经 src:公案
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import sys
|
import sys
|
||||||
import os
|
import os
|
||||||
import duckdb
|
import sqlite3
|
||||||
|
|
||||||
DB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "termlib.duckdb")
|
DB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "termlib.sqlite")
|
||||||
|
|
||||||
|
|
||||||
def _connect():
|
def _connect():
|
||||||
return duckdb.connect(DB, read_only=True)
|
con = sqlite3.connect(DB)
|
||||||
|
con.execute("PRAGMA journal_mode=WAL")
|
||||||
|
return con
|
||||||
|
|
||||||
|
|
||||||
def _search_rows(con, query, loc=None, src=None, limit=None):
|
def _search_rows(con, query, loc=None, src=None, limit=None):
|
||||||
@@ -42,7 +47,7 @@ def _search_rows(con, query, loc=None, src=None, limit=None):
|
|||||||
where += " AND source = ?"
|
where += " AND source = ?"
|
||||||
params.append(src)
|
params.append(src)
|
||||||
|
|
||||||
sql = f"SELECT zh, en, loc, source FROM unified_terms_flat WHERE {where}"
|
sql = f"SELECT zh, en, loc, source FROM terms WHERE {where}"
|
||||||
if limit is not None:
|
if limit is not None:
|
||||||
sql += " LIMIT ?"
|
sql += " LIMIT ?"
|
||||||
params.append(limit)
|
params.append(limit)
|
||||||
|
|||||||
Regular → Executable
+16
-7
@@ -1,22 +1,31 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env -S uv run --script
|
||||||
|
# /// script
|
||||||
|
# dependencies = ["flask"]
|
||||||
|
# ///
|
||||||
import sys, os
|
import sys, os
|
||||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||||
|
|
||||||
import duckdb
|
import sqlite3
|
||||||
from flask import Flask, request, jsonify, render_template
|
from flask import Flask, request, jsonify, render_template
|
||||||
import search as s
|
import search as s
|
||||||
|
|
||||||
app = Flask(__name__)
|
app = Flask(__name__)
|
||||||
|
|
||||||
|
def _connect():
|
||||||
|
con = sqlite3.connect(s.DB)
|
||||||
|
con.execute("PRAGMA journal_mode=WAL")
|
||||||
|
return con
|
||||||
|
|
||||||
|
|
||||||
def do_search(q, loc, src, limit):
|
def do_search(q, loc, src, limit):
|
||||||
with duckdb.connect(s.DB, read_only=True) as con:
|
rows = s.search(q, loc=loc, src=src, limit=limit)
|
||||||
rows = s.search(con, q, loc, src, limit)
|
return [{'zh': r['zh'], 'en': r['en'], 'loc': r['loc'] or None, 'source': r['source']} for r in rows]
|
||||||
return [{'zh': r[0], 'en': r[1], 'loc': r[2] or None, 'source': r[3]} for r in rows]
|
|
||||||
|
|
||||||
def do_sources():
|
def do_sources():
|
||||||
with duckdb.connect(s.DB, read_only=True) as con:
|
with _connect() as con:
|
||||||
rows = con.execute(
|
rows = con.execute(
|
||||||
'SELECT source, COUNT(*) AS cnt FROM unified_terms_flat GROUP BY source ORDER BY cnt DESC'
|
'SELECT source, COUNT(*) AS cnt FROM terms GROUP BY source ORDER BY cnt DESC'
|
||||||
).fetchall()
|
).fetchall()
|
||||||
return [{'source': r[0], 'count': r[1]} for r in rows]
|
return [{'source': r[0], 'count': r[1]} for r in rows]
|
||||||
|
|
||||||
|
|||||||
Binary file not shown.
Binary file not shown.
Reference in New Issue
Block a user