Refactor codebase
This commit is contained in:
iacore
@@ -0,0 +1,3 @@
|
||||
[submodule "toolkit"]
|
||||
path = toolkit
|
||||
url = git@git.sr.ht:~iacore/mpi-translation-toolkit
|
||||
@@ -1,7 +0,0 @@
|
||||
# MPI Project Instructions
|
||||
|
||||
You are working on the Mindful Peace International Chinese-English Buddhist/Dharma translation project.
|
||||
- Before translating, load the `translation` and `terms-search` skills.
|
||||
- Before reviewing, load `self-review` (and `other-review` for peer review).
|
||||
- The agent IS the model: do not call external translation APIs.
|
||||
- The workflow is not as rigid as the state machine in AGENTS.md. The user may ask for you to deviate from the default workflow (described by the state machine). Be flexible when asked.
|
||||
@@ -1,196 +1,3 @@
|
||||
# MPI Project Conventions
|
||||
|
||||
## Skills
|
||||
|
||||
Skills in `skills/`.
|
||||
|
||||
Available: `translation`, `terms-search`, `translation-review`, `chinese-text-normalize`, `pptx-translate`, `pdf-to-docx-conversion`.
|
||||
|
||||
## Terms Database
|
||||
|
||||
See `terms-search` skill. Quick reference:
|
||||
- CLI: `terms-search/search.py <query> [limit]`
|
||||
- Module: `from search import search; search("空性", limit=5, src="DoT定稿")`
|
||||
- Priority: DoT定稿 > 内部特色词 > 佛教术语 > 经论名
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
translate-files/<topic>/<article>/
|
||||
source.dj — Chinese original
|
||||
target.dj — English translation (line count matches source)
|
||||
bilingual.dj — interleaved (source line, blank, target line, blank)
|
||||
Generated by `../../scripts/gen-bilingual.py source.dj target.dj`;
|
||||
do not edit or commit.
|
||||
```
|
||||
|
||||
Put `.docx` output in `/tmp/`. Don't commit binaries.
|
||||
Generated files (`bilingual.dj`) are not committed either.
|
||||
|
||||
---
|
||||
|
||||
## Translation State Machine
|
||||
|
||||
All translation work follows this deterministic workflow. Non-deterministic LLM work (drafting, reviewing) happens at the edges; the states and transitions are fixed.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> idle
|
||||
idle --> translating: SOURCE_LOADED
|
||||
idle --> other_reviewing: BILINGUAL_LOADED
|
||||
translating --> bilingual_ready: TRANSLATION_DRAFTED
|
||||
bilingual_ready --> self_reviewing: BILINGUAL_GENERATED
|
||||
self_reviewing --> translating: SELF_REJECTED
|
||||
self_reviewing --> other_reviewing: SELF_APPROVED [peer review required]
|
||||
self_reviewing --> approved: SELF_APPROVED [no peer review]
|
||||
note right of self_reviewing
|
||||
peer_review_required flag decides the branch
|
||||
end note
|
||||
other_reviewing --> translating: PEER_REJECTED
|
||||
other_reviewing --> approved: PEER_APPROVED
|
||||
approved --> typesetting: TYPESET_REQUESTED
|
||||
approved --> done: COMPLETE
|
||||
typesetting --> done: TYPESET_COMPLETE
|
||||
done --> [*]
|
||||
```
|
||||
|
||||
States:
|
||||
|
||||
| State | Meaning | Output artifact |
|
||||
|---|---|---|
|
||||
| `idle` | Waiting for source or an existing bilingual file. | — |
|
||||
| `translating` | Agent loads `translation` + `terms-search` skills and drafts `target.dj`. | `target.dj` |
|
||||
| `bilingual_ready` | `bilingual.dj` generated from `source.dj` + `target.dj`. | `bilingual.dj` |
|
||||
| `self_reviewing` | Self-review with unified ruleset (self mode). Edit target.dj. | `target.dj` (edited) |
|
||||
| `other_reviewing` | Peer review with unified ruleset (other mode). Write review-comments.dj. | `review-comments.dj` |
|
||||
| `approved` | Translation accepted. May typeset or finish. | — |
|
||||
| `typesetting` | Producing PDF/DOCX from approved bilingual content. | `.pdf` / `.docx` |
|
||||
| `done` | Complete. | — |
|
||||
|
||||
## Workflow A: Translation（翻译）
|
||||
|
||||
Translate Chinese source into English. The agent IS the model — no external APIs. This workflow covers the state machine path `idle` → `translating` → `bilingual_ready` → `self_reviewing`.
|
||||
|
||||
### Source context
|
||||
|
||||
Before translating, the agent must understand the source's format and delivery context. If the source is a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, a written article, or any other genre, that register shapes the translation. If this context is not clear from the file path or source content, ask the user before proceeding.
|
||||
|
||||
### Input
|
||||
|
||||
Source text in `.dj` or `.docx` (Chinese only).
|
||||
|
||||
### Deliverables
|
||||
|
||||
- `source.dj` — extracted/cleaned Chinese
|
||||
- `target.dj` — English translation, line count matches source
|
||||
- `bilingual.dj` — interleaved (source line, target line adjacent, blank between pairs).
|
||||
Generated by `../../scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||||
Do not create or edit by hand; do not commit.
|
||||
- `edit-suggestions.dj` — terminology/consistency issues flagged for review
|
||||
|
||||
### Rules
|
||||
|
||||
1. Load `translation` and `terms-search` skills before starting.
|
||||
2. Search terms DB for key Buddhist terms.
|
||||
3. TOC: plain bullet lists, no link targets, no page numbers.
|
||||
4. Djot formatting:
|
||||
- Emphasis: `*text*` (single asterisks). Never `**` (Markdown bold).
|
||||
- Comments: `{% ... %}`
|
||||
5. Preserve source formatting — don't add/remove emphasis.
|
||||
6. Translate in-response — never call external translation APIs.
|
||||
|
||||
### Review
|
||||
|
||||
After translating, load `translation-review` (self mode) to check:
|
||||
- Full detection rules (three passes + R1–R14 editorial polish)
|
||||
- Terminology consistency against terms DB
|
||||
- Grammar, fluency, calques
|
||||
- Missing content (mid-paragraph truncation)
|
||||
- Inconsistency (same term translated differently)
|
||||
|
||||
---
|
||||
|
||||
## Workflow B: Proofread / Review（校对/审阅）
|
||||
|
||||
Both self-review and other-review use the unified `translation-review` skill.
|
||||
The **same detection rules** apply to both. The only difference: self mode edits
|
||||
`target.dj` directly; other mode writes `review-comments.dj` with collaborative tone.
|
||||
|
||||
A third mode — **Direct Edit Review** — applies when the user explicitly says to
|
||||
edit the `.dj` files directly and skip any comments file.
|
||||
|
||||
### B1: Self-Review（自审）
|
||||
|
||||
You translated it. You own the English. Load `translation-review` skill (self mode).
|
||||
|
||||
1. Read `source.dj` + `target.dj` fully.
|
||||
2. Apply all detection rules (three passes + R1–R14 editorial polish).
|
||||
3. Edit `target.dj` directly with `patch` (mode='replace').
|
||||
4. Record non-obvious choices in `translation-findings.dj` if needed.
|
||||
5. Regenerate `bilingual.dj` with `../../scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||||
6. Verify line counts: `source.dj` and `target.dj` must match.
|
||||
|
||||
### B2: Other-Review（审他稿）
|
||||
|
||||
Someone else translated it (volunteer, etc.). Load `translation-review` skill (other mode).
|
||||
|
||||
1. Read `source.dj` + `target.dj` fully.
|
||||
2. Apply all detection rules (three passes + R1–R14 editorial polish).
|
||||
3. Do NOT edit `target.dj` — write `review-comments.dj` instead.
|
||||
4. Follow deliberation protocol: 随喜 first, questions not commands.
|
||||
5. Address translator by name.
|
||||
6. Ask the user whether to apply the findings. If yes, switch to Direct Edit Mode.
|
||||
|
||||
### B3: Direct Edit Review（直接修改稿）
|
||||
|
||||
The user wants fixes applied directly, no separate comments file. This can follow
|
||||
self-review or other-review, or be a standalone polish pass.
|
||||
|
||||
1. Read full `target.dj` + `source.dj` in one pass.
|
||||
2. Collect every issue using the full detection rules.
|
||||
3. Batch all fixes into one set of exact-string replacements. Apply with `patch`.
|
||||
4. Regenerate `bilingual.dj` with `../../scripts/gen-bilingual.py source.dj target.dj > bilingual.dj`.
|
||||
5. Verify line counts match.
|
||||
|
||||
Avoid iterative "find a few more, edit again" loops. If the user asks "anything
|
||||
else?" after a direct-edit pass, do one more full systematic read and batch again.
|
||||
|
||||
---
|
||||
|
||||
## Djot
|
||||
|
||||
- Comments: `{% ... %}`
|
||||
- Emphasis: `*text*` (single asterisks)
|
||||
- Dashes in English: `---` em, `--` en. Pandoc converts in docx output.
|
||||
- Preserve source formatting — don't add/remove emphasis
|
||||
|
||||
When editing `.dj` files, use `patch` (mode='replace') — not regex-based
|
||||
string replacement in `execute_code`. `patch` is safer, surfaces conflicts,
|
||||
and produces a diff you can review.
|
||||
|
||||
## Typst Bilingual Template
|
||||
|
||||
For producing PDFs from bilingual Chinese-English articles:
|
||||
|
||||
- Template: `translate-files/lib/mpi-bilingual-template.typ`
|
||||
- Example host: `translate-files/从物品整理到心灵整理/mindful-organizing.typ`
|
||||
- Design notes: `references/typst-template-design.md`
|
||||
- Produce rendered PDF files in: /tmp/
|
||||
|
||||
## Scripts
|
||||
|
||||
Utility scripts in `scripts/` (fish for CLI wrappers, Python for data processing).
|
||||
Agents should write repetitive logic here and run via `terminal` rather than
|
||||
regenerating the same Python in execute_code each turn.
|
||||
|
||||
- `scripts/docx2dj.fish <docx>` — pandoc .docx → .dj alongside the original
|
||||
- `scripts/split-bilingual.fish <combined.dj>` — split into source.dj (CN) + target.dj (EN)
|
||||
- `scripts/dj2docx.fish <target.dj>` — pandoc .dj → .docx in `/tmp/`
|
||||
- `scripts/proofread-pdf.py <docx> <pdf>` — word-level diff between manuscript and typeset PDF
|
||||
- `scripts/gen-bilingual.py <source.dj> <target.dj>` — produce `bilingual.dj` on stdout; run as `gen-bilingual.py source.dj target.dj > bilingual.dj`
|
||||
- `scripts/gen-bilingual-<name>-<hash>.py` — article-specific extraction from DOCX or source/target pairing
|
||||
- `scripts/compile-typst.fish <typ>` — compile a Typst file to PDF
|
||||
|
||||
Article-specific scripts (including Typst compile helpers) should be placed in
|
||||
the article directory itself, named with a short hash: e.g.
|
||||
`translate-files/<article>/compile-typst-<hash>.fish`.
|
||||
See [toolkit/AGENTS.md](toolkit/AGENTS.md) for the full project conventions.
|
||||
|
||||
@@ -0,0 +1,513 @@
|
||||
# Using omp to Translate Articles
|
||||
|
||||
A beginner's guide for the MPI Translation Project.
|
||||
|
||||
This guide assumes you have never used omp or an AI coding agent before. We will walk through every step: downloading omp, buying AI credits, connecting a provider, and running your first translation.
|
||||
|
||||
---
|
||||
|
||||
## What this guide will teach you
|
||||
|
||||
By the end of this guide you will be able to:
|
||||
|
||||
- Install omp on your computer.
|
||||
- Set up an AI model provider with credits.
|
||||
- Connect that provider to omp.
|
||||
- Open a translation project and start translating a Chinese article into English.
|
||||
- Understand the basic workflow from source to approved translation.
|
||||
|
||||
---
|
||||
|
||||
## What you need before you start
|
||||
|
||||
- A computer running macOS, Linux, or Windows.
|
||||
- A terminal (on macOS or Linux) or PowerShell / Windows Terminal (on Windows).
|
||||
- An internet connection.
|
||||
- A payment method for AI credits: credit card, debit card, or the payment options each provider accepts.
|
||||
- A copy of the translation project files on your computer.
|
||||
|
||||
No programming experience is required. You will mostly be typing plain commands and reading the output.
|
||||
|
||||
---
|
||||
|
||||
## What is omp?
|
||||
|
||||
omp ("Oh My Pi") is a terminal-first AI assistant --- a *coding agent* --- that runs on your computer. It reads files, runs commands, edits text, and follows detailed instructions you give it.
|
||||
|
||||
For translation work, you use omp to:
|
||||
|
||||
- Read the Chinese source file.
|
||||
- Look up Buddhist terms in the project's term database.
|
||||
- Draft an English translation.
|
||||
- Review the draft for mistakes and awkward wording.
|
||||
- Generate side-by-side bilingual files.
|
||||
|
||||
You stay in charge. omp drafts and checks; you approve and correct.
|
||||
|
||||
---
|
||||
|
||||
## What is an AI model provider?
|
||||
|
||||
omp is the assistant. The *provider* is the AI service that actually thinks and writes. The provider charges you for the compute time used.
|
||||
|
||||
For this project we recommend one of these three providers:
|
||||
|
||||
| Provider | Best for | Pricing model |
|
||||
|---|---|---|
|
||||
| OpenCode Go | Low-cost, predictable monthly access to open coding models. | $5 first month, then $10/month. |
|
||||
| OpenRouter | Pay-as-you-go access to many models, including Claude and GPT. | Buy credits; no minimum spend. |
|
||||
| Kimi Code | Access to Kimi's coding models. | Recharge API credits; starts at $1. |
|
||||
|
||||
You only need *one* of these to get started. Pick the one that is easiest to pay for in your region.
|
||||
|
||||
---
|
||||
|
||||
## Step 1: Install omp
|
||||
|
||||
### macOS and Linux
|
||||
|
||||
Open your terminal and run one of these commands.
|
||||
|
||||
**Recommended --- installer script:**
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/can1357/oh-my-pi/main/scripts/install.sh | sh
|
||||
```
|
||||
|
||||
**If you use Homebrew:**
|
||||
|
||||
```bash
|
||||
brew install can1357/tap/omp
|
||||
```
|
||||
|
||||
**If you already have Bun installed (version 1.3.14 or newer):**
|
||||
|
||||
```bash
|
||||
bun install -g @oh-my-pi/pi-coding-agent
|
||||
```
|
||||
|
||||
### Windows
|
||||
|
||||
Open PowerShell and run:
|
||||
|
||||
```powershell
|
||||
irm https://omp.sh/install.ps1 | iex
|
||||
```
|
||||
|
||||
### Verify the installation
|
||||
|
||||
Close and reopen your terminal, then run:
|
||||
|
||||
```bash
|
||||
omp --version
|
||||
```
|
||||
|
||||
You should see a version number. If you see an error like "command not found," check that your terminal was restarted after installation.
|
||||
|
||||
Also run:
|
||||
|
||||
```bash
|
||||
omp config path
|
||||
```
|
||||
|
||||
This shows where omp keeps its settings, usually `~/.omp/agent/`. You will use this folder later.
|
||||
|
||||
---
|
||||
|
||||
## Step 2: Get AI access
|
||||
|
||||
Choose **one** provider below and follow the steps. You do not need all three.
|
||||
|
||||
---
|
||||
|
||||
### Option A: OpenCode Go
|
||||
|
||||
OpenCode Go is a subscription that gives you reliable access to several open coding models for a flat monthly price.
|
||||
|
||||
1. Open a browser and go to `https://opencode.ai/auth`.
|
||||
2. Create an account with email or a third-party login.
|
||||
3. Subscribe to **OpenCode Go**.
|
||||
- The first month costs $5.
|
||||
- After the first month it costs $10/month.
|
||||
4. Once you are subscribed, find the API key in your account page.
|
||||
5. Copy the key and save it somewhere safe. Treat it like a password.
|
||||
|
||||
Models included with OpenCode Go include GLM-5.2, Kimi K2.7 Code, Kimi K2.6, Qwen3.7 Max, DeepSeek V4 Pro, and others. The exact list can change; see `https://opencode.ai/docs/go` for the current models.
|
||||
|
||||
---
|
||||
|
||||
### Option B: OpenRouter
|
||||
|
||||
OpenRouter is a gateway that lets you use many different AI models through a single account. You pay only for what you use.
|
||||
|
||||
1. Open a browser and go to `https://openrouter.ai/sign-up`.
|
||||
2. Create an account with GitHub, Google, MetaMask, or email.
|
||||
3. Sign in, then go to `https://openrouter.ai/settings/credits`.
|
||||
4. Click **Buy Credits** and choose an amount. There is no minimum spend; $10 is enough to start.
|
||||
5. Add a payment method and complete the purchase.
|
||||
6. Go to `https://openrouter.ai/keys` and create an API key.
|
||||
7. Copy the key and save it somewhere safe. Treat it like a password.
|
||||
|
||||
OpenRouter charges a 5.5% platform fee on top of the model's price. You can explore models and prices at `https://openrouter.ai/models`.
|
||||
|
||||
---
|
||||
|
||||
### Option C: Kimi Code
|
||||
|
||||
Kimi Code is the coding model from Moonshot AI. You use the Kimi API platform to create an account and recharge credits.
|
||||
|
||||
1. Open a browser and go to `https://platform.kimi.ai/`.
|
||||
2. Create an account and sign in.
|
||||
3. Go to the console or user center.
|
||||
4. Recharge your account.
|
||||
- You need at least $1 to start using the API.
|
||||
- When your cumulative recharge reaches $5, you receive a $5 voucher.
|
||||
5. Go to the API key section and create a key.
|
||||
6. Copy the key and save it somewhere safe. Treat it like a password.
|
||||
|
||||
Pricing for each model is listed at `https://platform.kimi.ai/docs/pricing/chat`. For coding, the relevant model is usually **Kimi K2.7 Code**. You are charged per token used.
|
||||
|
||||
---
|
||||
|
||||
## Step 3: Connect your provider to omp
|
||||
|
||||
omp reads your provider key from an environment variable or a `.env` file. The easiest way to start is with a `.env` file in your home directory.
|
||||
|
||||
### Create the `.env` file
|
||||
|
||||
Run this in your terminal to open or create the file:
|
||||
|
||||
```bash
|
||||
nano ~/.omp/.env
|
||||
```
|
||||
|
||||
If you prefer a different editor, replace `nano` with `vim`, `code`, or another editor.
|
||||
|
||||
Paste the block for the provider you chose:
|
||||
|
||||
**OpenCode Go:**
|
||||
|
||||
```bash
|
||||
OPENCODE_API_KEY=opencode-your-key-here
|
||||
```
|
||||
|
||||
**OpenRouter:**
|
||||
|
||||
```bash
|
||||
OPENROUTER_API_KEY=sk-or-v1-your-key-here
|
||||
```
|
||||
|
||||
**Kimi Code:**
|
||||
|
||||
```bash
|
||||
KIMI_API_KEY=your-kimi-key-here
|
||||
```
|
||||
|
||||
Replace the placeholder after the `=` with the real API key you copied. Save the file and close the editor.
|
||||
|
||||
### Restart your terminal
|
||||
|
||||
Close and reopen your terminal so the new `.env` file is read. Then verify that omp launches:
|
||||
|
||||
```bash
|
||||
omp
|
||||
```
|
||||
|
||||
You should see the omp welcome screen. Press `Ctrl + C` to exit.
|
||||
|
||||
### Test with a simple prompt
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
omp -p "hello"
|
||||
```
|
||||
|
||||
If everything is connected, omp will reply with a short greeting. If you see an authentication error, check that your API key is pasted correctly and that the terminal was restarted.
|
||||
|
||||
---
|
||||
|
||||
## Step 4: Open your translation project
|
||||
|
||||
omp works inside a project folder. The project folder contains the Chinese source, the English translation, and the rules omp should follow.
|
||||
|
||||
In your terminal, move to the project folder. For example:
|
||||
|
||||
```bash
|
||||
cd ~/documents/mpi/other/omp-translation-guide
|
||||
```
|
||||
|
||||
Or, if you are working on a specific article:
|
||||
|
||||
```bash
|
||||
cd ~/documents/mpi/translate-files/<topic>/<article>
|
||||
```
|
||||
|
||||
Once you are inside the project folder, launch omp:
|
||||
|
||||
```bash
|
||||
omp
|
||||
```
|
||||
|
||||
omp will detect the project files and rules. The current folder becomes the project root.
|
||||
|
||||
---
|
||||
|
||||
## Step 5: Translate your first article
|
||||
|
||||
Inside an active omp session, you can ask the agent to translate. A typical first prompt looks like this:
|
||||
|
||||
```text
|
||||
Please translate this article from Chinese to English following the project conventions. Read source.dj, look up key Buddhist terms in the terms database, and produce target.dj.
|
||||
```
|
||||
|
||||
omp will:
|
||||
|
||||
1. Read `source.dj` (the Chinese original).
|
||||
2. Look up Buddhist terms in the MPI term database.
|
||||
3. Draft `target.dj` (the English translation).
|
||||
4. Generate `bilingual.dj` (a side-by-side file) if you ask for it.
|
||||
|
||||
You can watch the process in the terminal. Each step is shown as a compact card. Press `Ctrl + O` to expand a card and see the full output.
|
||||
|
||||
---
|
||||
|
||||
## The translation workflow
|
||||
|
||||
All translation work in this project follows a fixed state machine. The AI handles the drafting and review stages; humans approve the result.
|
||||
|
||||
```mermaid
|
||||
stateDiagram-v2
|
||||
[*] --> idle
|
||||
idle --> translating: SOURCE_LOADED
|
||||
idle --> other_reviewing: BILINGUAL_LOADED
|
||||
translating --> bilingual_ready: TRANSLATION_DRAFTED
|
||||
bilingual_ready --> self_reviewing: BILINGUAL_GENERATED
|
||||
self_reviewing --> translating: SELF_REJECTED
|
||||
self_reviewing --> other_reviewing: SELF_APPROVED [peer review required]
|
||||
self_reviewing --> approved: SELF_APPROVED [no peer review]
|
||||
other_reviewing --> translating: PEER_REJECTED
|
||||
other_reviewing --> approved: PEER_APPROVED
|
||||
approved --> typesetting: TYPESET_REQUESTED
|
||||
approved --> done: COMPLETE
|
||||
typesetting --> done: TYPESET_COMPLETE
|
||||
done --> [*]
|
||||
```
|
||||
|
||||
States:
|
||||
|
||||
| State | Meaning | Output artifact |
|
||||
|---|---|---|
|
||||
| `idle` | Waiting for source or an existing bilingual file. | --- |
|
||||
| `translating` | AI drafts the English translation. | `target.dj` |
|
||||
| `bilingual_ready` | `bilingual.dj` is generated from source and target. | `bilingual.dj` |
|
||||
| `self_reviewing` | AI checks its own draft against the source. | edited `target.dj` |
|
||||
| `other_reviewing` | A peer reviews the draft. | `review-comments.dj` |
|
||||
| `approved` | Translation accepted. | --- |
|
||||
| `typesetting` | Producing PDF or DOCX. | `.pdf` / `.docx` |
|
||||
| `done` | Complete. | --- |
|
||||
|
||||
For a total beginner, the most important path is:
|
||||
|
||||
1. **Prepare the source** --- clean the Chinese and save it as `source.dj`.
|
||||
2. **Draft** --- ask omp to create `target.dj`.
|
||||
3. **Generate bilingual** --- create `bilingual.dj` to read both languages side by side.
|
||||
4. **Self-review** --- ask omp to check its own draft for errors.
|
||||
5. **Human review** --- you read the bilingual file and approve or flag issues.
|
||||
|
||||
---
|
||||
|
||||
## Step 6: Prepare the source
|
||||
|
||||
The source usually arrives as a Word document or a plain-text file. We clean it and convert it into a Djot file called `source.dj`.
|
||||
|
||||
If you have a `.docx` file, run:
|
||||
|
||||
```bash
|
||||
toolkit/scripts/docx2dj.fish <article>.docx
|
||||
```
|
||||
|
||||
If you do not have a `.docx` file, you can create `source.dj` by hand in a text editor. Keep the line structure intact: one physical line per logical line, and preserve paragraph breaks.
|
||||
|
||||
Before translating, understand the source's genre. Is it a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, or a written article? The genre shapes the translation register. If the context is unclear, ask the team before proceeding.
|
||||
|
||||
---
|
||||
|
||||
## Step 7: Draft the translation
|
||||
|
||||
Once `source.dj` exists, ask omp to draft the translation. A good prompt is:
|
||||
|
||||
```text
|
||||
Load the mpi-translation and mpi-terms-search skills. Translate source.dj into English, preserving the line structure, and write target.dj. Look up key Buddhist terms in the MPI terms database before translating.
|
||||
```
|
||||
|
||||
omp will read the Chinese line by line, check the terms database, and write one English line for every Chinese line.
|
||||
|
||||
The terms database is searched with:
|
||||
|
||||
```bash
|
||||
$MPI_PROJECT_ROOT/toolkit/terms-database/search.py <term> [limit]
|
||||
```
|
||||
|
||||
Source priority is:
|
||||
|
||||
1. DoT定稿
|
||||
2. 内部特色词
|
||||
3. 佛教术语
|
||||
4. 经论名
|
||||
|
||||
omp also preserves emphasis markers (`*text*`) and soft-line markers (` ` at the end of a line).
|
||||
|
||||
---
|
||||
|
||||
## Step 8: Generate the bilingual file
|
||||
|
||||
Once `source.dj` and `target.dj` exist, generate the side-by-side file:
|
||||
|
||||
```bash
|
||||
../../toolkit/scripts/gen-bilingual.py source.dj target.dj > bilingual.dj
|
||||
```
|
||||
|
||||
The bilingual file interleaves source and target lines with a blank line between each pair. Do not edit `bilingual.dj` by hand. Regenerate it whenever `source.dj` or `target.dj` changes.
|
||||
|
||||
---
|
||||
|
||||
## Step 9: Self-review
|
||||
|
||||
Ask omp to review its own draft:
|
||||
|
||||
```text
|
||||
Load the mpi-translation-review skill in self mode. Read source.dj and target.dj fully, check for missing content, terminology consistency, and awkward English, then edit target.dj directly. After editing, regenerate bilingual.dj and verify the line counts match.
|
||||
```
|
||||
|
||||
The review runs three passes:
|
||||
|
||||
1. **Accuracy and completeness** --- missing content, mistranslation, terminology inconsistency, unnecessary additions, number/time/person mismatches.
|
||||
2. **Fluency and naturalness** --- calques, register drift, broken collocations, pronoun errors, sentence rhythm.
|
||||
3. **Dharma and cultural fitness** --- Buddhist term register, cultural anachronism, tone of the teacher, implicit meaning, formatting fidelity.
|
||||
|
||||
After the three passes, it applies an R1--R14 editorial checklist covering spelling, punctuation, capitalization, articles, agreement, tense, voice, prepositions, modifiers, parallelism, redundancy, word choice, sentence openings, flow, and a final read-aloud.
|
||||
|
||||
In self mode, omp edits `target.dj` directly and then regenerates `bilingual.dj`. It verifies that `source.dj` and `target.dj` have exactly the same number of lines.
|
||||
|
||||
---
|
||||
|
||||
## Step 10: Human review and peer review
|
||||
|
||||
Read `bilingual.dj` carefully. You are checking that the English matches the Chinese, sounds natural, and keeps the right tone.
|
||||
|
||||
If you want another person to review, they can ask omp to write a `review-comments.dj` file:
|
||||
|
||||
```text
|
||||
Load the mpi-translation-review skill in other mode. Read source.dj and target.dj and write review-comments.dj with must-fix issues and optional suggestions. Do not edit target.dj directly.
|
||||
```
|
||||
|
||||
The peer review follows a deliberation protocol: begin with appreciation, phrase most issues as questions or options, and distinguish "must fix" from "consider."
|
||||
|
||||
When the team asks omp to apply the findings, it switches to direct-edit mode, updates `target.dj`, and regenerates `bilingual.dj`.
|
||||
|
||||
---
|
||||
|
||||
## Step 11: Typesetting and publishing
|
||||
|
||||
Once the translation is approved, it can be exported.
|
||||
|
||||
- **DOCX export** using `toolkit/scripts/dj2docx.fish <target.dj>` produces a Word document in `/tmp/`.
|
||||
- **Bilingual PDF** using the Typst template `translate-files/lib/mpi-bilingual-template.typ` produces a PDF through `toolkit/scripts/compile-typst.fish <typ>`.
|
||||
|
||||
Rendered files are written to `/tmp/` and are not committed to the repository.
|
||||
|
||||
---
|
||||
|
||||
## Djot conventions
|
||||
|
||||
All translation files are written in Djot. Keep these rules in mind:
|
||||
|
||||
- Emphasis: use a single asterisk on each side: `*text*`. Never use `**` for bold.
|
||||
- Comments: wrap notes in `{% ... %}`.
|
||||
- Dashes: use `---` for an em dash and `--` for an en dash.
|
||||
- Preserve the source formatting. Do not add or remove emphasis.
|
||||
- Keep the one-to-one line mapping: one physical target line per physical source line.
|
||||
- Preserve soft-line markers (` ` at the end of a line) on both source and target.
|
||||
- Table of contents: plain bullet lists only, no link targets and no page numbers.
|
||||
|
||||
---
|
||||
|
||||
## Useful omp commands for translation
|
||||
|
||||
| Command | What it does |
|
||||
|---|---|
|
||||
| `omp` | Start an interactive session in the current project. |
|
||||
| `omp -p "<prompt>"` | Run a single prompt and exit. |
|
||||
| `Ctrl + O` | Expand the selected tool card to see full output. |
|
||||
| `Ctrl + C` | Exit omp. |
|
||||
| `/skill:<name>` | Load a specific skill manually. |
|
||||
| `/model` | Pick a different model from the providers you are signed into. |
|
||||
|
||||
---
|
||||
|
||||
## Useful scripts
|
||||
|
||||
| Script | Purpose |
|
||||
|---|---|
|
||||
| `toolkit/scripts/docx2dj.fish <docx>` | Convert a Word document to Djot. |
|
||||
| `toolkit/scripts/split-bilingual.fish <combined.dj>` | Split a combined bilingual file into `source.dj` and `target.dj`. |
|
||||
| `toolkit/scripts/dj2docx.fish <target.dj>` | Convert a Djot translation to DOCX in `/tmp/`. |
|
||||
| `toolkit/scripts/gen-bilingual.py <source.dj> <target.dj>` | Generate `bilingual.dj` on stdout. |
|
||||
| `toolkit/scripts/compile-typst.fish <typ>` | Compile a Typst file to PDF. |
|
||||
|
||||
Article-specific scripts are placed in the article directory itself and named with a short hash.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "omp: command not found"
|
||||
|
||||
- Restart your terminal after installation.
|
||||
- Check that the install directory is on your `PATH`. Common locations are `~/.local/bin` and `/usr/local/bin`.
|
||||
|
||||
### Authentication error
|
||||
|
||||
- Double-check that your API key is pasted correctly in `~/.omp/.env`.
|
||||
- Make sure you restarted your terminal after saving the `.env` file.
|
||||
- Make sure the key has not expired.
|
||||
|
||||
### No response or very slow response
|
||||
|
||||
- Check your internet connection.
|
||||
- Check the provider's status page (for example, `https://status.openrouter.ai/`).
|
||||
- Switch to a different model with `/model` if the current model is overloaded.
|
||||
|
||||
### Translation line counts do not match
|
||||
|
||||
- Run `wc -l source.dj target.dj` to see which file is longer.
|
||||
- Ask omp to regenerate `target.dj` or `bilingual.dj` from the current source.
|
||||
- Check that no blank lines were accidentally added or removed.
|
||||
|
||||
---
|
||||
|
||||
## Quick checklist
|
||||
|
||||
Before declaring a translation ready for human review, confirm:
|
||||
|
||||
- [ ] omp is installed and `omp --version` works.
|
||||
- [ ] An AI provider is connected and `omp -p "hello"` replies.
|
||||
- [ ] `source.dj` and `target.dj` have the same number of lines.
|
||||
- [ ] `bilingual.dj` has been regenerated from the latest source and target.
|
||||
- [ ] Key Buddhist terms have been checked in the MPI terms database.
|
||||
- [ ] No missing content, truncation, or overtranslation.
|
||||
- [ ] The English matches the source's genre and register.
|
||||
- [ ] Emphasis, paragraph breaks, and soft-line markers are preserved.
|
||||
- [ ] The final English has been read aloud.
|
||||
|
||||
---
|
||||
|
||||
## Next steps
|
||||
|
||||
- Read the `mpi-translation` skill for the full translation principles.
|
||||
- Read the `mpi-terms-search` skill for how to query the MPI term database.
|
||||
- Read the `mpi-translation-review` skill for the three-pass review and R1--R14 checklist.
|
||||
- Read `AGENTS.md` for the project conventions and file layout.
|
||||
|
||||
When you feel comfortable, try translating a short article from start to finish. The best way to learn is by doing.
|
||||
@@ -0,0 +1,249 @@
|
||||
#import "@preview/touying:0.7.4": *
|
||||
#import themes.simple: *
|
||||
|
||||
#let warm-accent = rgb("#A0522D")
|
||||
#let dark-text = rgb("#333333")
|
||||
#let soft-bg = rgb("#FDFBF7")
|
||||
|
||||
#show: simple-theme.with(
|
||||
aspect-ratio: "16-9",
|
||||
config-colors(
|
||||
primary: warm-accent,
|
||||
secondary: dark-text,
|
||||
tertiary: rgb("#8B7355"),
|
||||
neutral: dark-text,
|
||||
neutral-lightest: soft-bg,
|
||||
),
|
||||
)
|
||||
|
||||
#set text(font: "Liberation Sans", size: 22pt, fill: dark-text)
|
||||
#set heading(numbering: none)
|
||||
|
||||
// Custom title slide so title and subtitle appear together
|
||||
#slide[
|
||||
#align(center + horizon)[
|
||||
#text(size: 2em, weight: "bold", fill: dark-text)[
|
||||
How we translate Buddhist articles
|
||||
]
|
||||
#v(1em)
|
||||
#text(size: 1.2em, fill: dark-text)[
|
||||
From Chinese source to English reader
|
||||
]
|
||||
#v(0.5em)
|
||||
#text(size: 1em, fill: warm-accent)[
|
||||
with a little help from AI
|
||||
]
|
||||
#v(2em)
|
||||
#text(size: 0.8em, fill: rgb("#666666"))[
|
||||
MPI Translation Project · omp
|
||||
]
|
||||
]
|
||||
]
|
||||
|
||||
== What is omp?
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
"omp" stands for *Oh My Pi*.
|
||||
|
||||
#pause
|
||||
|
||||
Think of it as a shared workspace where AI assistants help us with careful, repetitive work.
|
||||
|
||||
#pause
|
||||
|
||||
- We give the AI our rules, our files, and our style guide.
|
||||
- It follows the same process every time.
|
||||
- We stay in charge: AI drafts, humans review.
|
||||
|
||||
== What we have translated so far
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
Our library holds *14 article projects* on Buddhist and Dharma topics.
|
||||
|
||||
#pause
|
||||
|
||||
So far, *2 articles* are fully translated:
|
||||
|
||||
#pause
|
||||
|
||||
- *What do we rely on to understand the world?*
|
||||
- *12-Minute Mindful Ball Guidance*
|
||||
|
||||
#pause
|
||||
|
||||
More articles are in progress, covering mindfulness, life questions, gratitude, end-of-life care, and Dharma in management.
|
||||
|
||||
== Our specialized helpers
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
Inside omp, we keep special guidebooks called *skills*. They teach the AI how to do each task.
|
||||
|
||||
#pause
|
||||
|
||||
#grid(
|
||||
columns: (1fr, 1fr),
|
||||
gutter: 1.2em,
|
||||
[
|
||||
*Translation skill*
|
||||
|
||||
How to turn Chinese into English
|
||||
],
|
||||
[
|
||||
*Terms-search skill*
|
||||
|
||||
How to look up Buddhist terms in our database
|
||||
],
|
||||
[
|
||||
*Review skill*
|
||||
|
||||
How to check and improve drafts
|
||||
],
|
||||
[
|
||||
*Other helpers*
|
||||
|
||||
Clean up text, translate slides, convert documents
|
||||
],
|
||||
)
|
||||
|
||||
== Step 1: Receiving the article
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
An article usually arrives as a Word document or a plain text file.
|
||||
|
||||
#pause
|
||||
|
||||
We clean it up and prepare a master Chinese file called `source.dj`.
|
||||
|
||||
#pause
|
||||
|
||||
This master copy becomes the original that every later step checks against.
|
||||
|
||||
== Step 2: First draft
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
The AI reads the Chinese line by line.
|
||||
|
||||
#pause
|
||||
|
||||
It looks up key Buddhist terms in our database so names and ideas stay consistent.
|
||||
|
||||
#pause
|
||||
|
||||
Then it writes a first English draft, called `target.dj`, with one English line for every Chinese line.
|
||||
|
||||
== Step 3: Self-review
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
A second AI checks the English draft against the original Chinese.
|
||||
|
||||
#pause
|
||||
|
||||
It looks for:
|
||||
|
||||
#pause
|
||||
|
||||
- missing sentences or paragraphs
|
||||
- awkward English
|
||||
- Buddhist terms used differently from before
|
||||
- anything that changes the meaning
|
||||
|
||||
#pause
|
||||
|
||||
It edits the English directly, and we check the line counts match.
|
||||
|
||||
== Step 4: Human review
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
Now we read the Chinese and English side by side.
|
||||
|
||||
#pause
|
||||
|
||||
#grid(
|
||||
columns: (1fr, 1fr),
|
||||
gutter: 1em,
|
||||
[
|
||||
#align(center, text(size: 1.2em)[中文原文])
|
||||
|
||||
#align(center, text(size: 0.8em)[Chinese original])
|
||||
],
|
||||
[
|
||||
#align(center, text(size: 1.2em)[English])
|
||||
|
||||
#align(center, text(size: 0.8em)[translation])
|
||||
],
|
||||
)
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
#pause
|
||||
|
||||
We approve the draft or flag problems for the AI to fix. This step may repeat several times until the text feels right.
|
||||
|
||||
== Step 5: Review by others
|
||||
|
||||
#v(0.5em)
|
||||
|
||||
When the translation is clean, we send it to volunteers or editors.
|
||||
|
||||
#pause
|
||||
|
||||
They read it with fresh eyes.
|
||||
|
||||
#pause
|
||||
|
||||
Once they approve, the article is ready for the next step: typesetting or publishing.
|
||||
|
||||
== The whole journey
|
||||
|
||||
#v(1em)
|
||||
|
||||
#text(size: 1.1em)[
|
||||
1. Receive the article
|
||||
|
||||
2. Draft the translation
|
||||
|
||||
3. Self-review
|
||||
|
||||
4. Human review
|
||||
|
||||
5. Review by others
|
||||
|
||||
6. Publish
|
||||
]
|
||||
|
||||
#v(1em)
|
||||
|
||||
#text(size: 0.85em, fill: rgb("#666666"))[
|
||||
AI helps at the first three stages; humans lead the last two.
|
||||
]
|
||||
|
||||
== Any questions?
|
||||
|
||||
#v(2em)
|
||||
|
||||
#align(center, text(size: 1.3em)[We would love to hear from you.])
|
||||
|
||||
#v(1em)
|
||||
|
||||
#align(center, text(size: 0.9em, fill: rgb("#666666"))[
|
||||
Ask about the workflow, the AI, or how you can help review translations.
|
||||
])
|
||||
|
||||
== Thank you
|
||||
|
||||
#v(2.5em)
|
||||
|
||||
#align(center, text(size: 1.5em)[Thank you for listening.])
|
||||
|
||||
#v(1em)
|
||||
|
||||
#align(center, text(size: 0.9em, fill: rgb("#666666"))[
|
||||
MPI Translation Project · omp
|
||||
])
|
||||
@@ -4,12 +4,13 @@
|
||||
|
||||
| Directory | Contents |
|
||||
|---|---|
|
||||
| `skills/` | Hermes Agent skills — loaded via `~/.hermes/config.yaml` `external_dirs`. [Install guide →](skills/readme.dj) |
|
||||
| `terms-search/` | Term database (DuckDB) + search CLI (`search.py`). Priority: DoT定稿 > 内部特色词 > 佛教术语 |
|
||||
| `toolkit/skills/` | Hermes Agent skills — loaded via `~/.hermes/config.yaml` `external_dirs`. [Install guide →](toolkit/skills/readme.dj) |
|
||||
| `toolkit/terms-database/` | Term database (DuckDB) + search CLI (`search.py`). Priority: DoT定稿 > 内部特色词 > 佛教术语 |
|
||||
| `translate-files/` | Translation projects: `<topic>/<article>/` with `source.dj`, `target.dj`, `bilingual.dj` |
|
||||
| `scripts/` | Utility scripts (fish shell): `dj2docx.fish` |
|
||||
| `guide/` | Reference materials — 术语库 source data |
|
||||
| `AGENTS.md` | Project conventions for AI agents — translation workflow, djot rules, skill usage |
|
||||
| `toolkit/scripts/` | Utility scripts (fish shell): `dj2docx.fish` |
|
||||
| `references/` | Reference materials — Typst design notes, Google Docs review journey, formatting guides |
|
||||
| `toolkit/` | Git submodule containing skills, scripts, term database, and AGENTS.md |
|
||||
| `AGENTS.md` | Stub pointing to `toolkit/AGENTS.md` |
|
||||
|
||||
## Translation Stages
|
||||
|
||||
|
||||
@@ -0,0 +1,133 @@
|
||||
# Google Docs Comment Automation — Journey Log
|
||||
|
||||
2026-06-21
|
||||
|
||||
## Background
|
||||
|
||||
Task: review Chinese→English Buddhist translation manuscripts on Google Docs
|
||||
by adding editorial comments (the "Wade" style — anchored suggestions on
|
||||
specific text). The reviewer is an AI agent running on Hermes with API access.
|
||||
|
||||
## Phase 1: Google Workspace OAuth Setup
|
||||
|
||||
Set up OAuth2 for Google Workspace access:
|
||||
- Created Google Cloud project (932146571366)
|
||||
- Enabled APIs: Gmail, Calendar, Drive, Sheets, Docs, People
|
||||
- Created Desktop OAuth client, downloaded client_secret.json
|
||||
- Completed OAuth PKCE flow → token at ~/.hermes/google_token.json
|
||||
|
||||
Success: authenticated with scopes covering all needed services.
|
||||
|
||||
## Phase 2: Reading Documents and Comments (Success)
|
||||
|
||||
Used the Google Drive API v3 `comments.list` endpoint to fetch comments
|
||||
from existing review documents:
|
||||
|
||||
- Doc 1 (1DTzH7...): 61 人生佛教在当代的弘扬 — 121 comments
|
||||
- Doc 2 (13S_0u...): 附录 我的判教观 — 151 comments
|
||||
- Doc 3 (1LxCcX...): 16 觉醒的艺术 — 201 comments
|
||||
|
||||
Total: 473 editor comments by Wade, spanning 2026-04-17 to 2026-06-20.
|
||||
These comments were well-anchored (kix.XXXXXXXX format) and contained
|
||||
actionable editorial suggestions in Chinese.
|
||||
|
||||
## Phase 3: Pattern Analysis (Success)
|
||||
|
||||
Analyzed all 473 comments to extract systematic review rules. The editor's
|
||||
feedback follows clear patterns:
|
||||
|
||||
R1 — Active voice + "we" subject
|
||||
R2 — Noun → verb conversion
|
||||
R3 — Simplify vocabulary
|
||||
R4 — Break long sentences
|
||||
R5 — Remove unnecessary words
|
||||
R6 — Conversational / interview tone
|
||||
R7 — No -ly adverbs
|
||||
R8 — Concrete over abstract
|
||||
R9 — Terminology alignment
|
||||
R10 — Missing content detection
|
||||
R11 — Source faithfulness
|
||||
R12 — Sentence structure clarity
|
||||
R13 — Specific word fixes
|
||||
R14 — Positive feedback (随喜), naming the translator
|
||||
|
||||
Created skill: toolkit/skills/mpi-translation-review-comment/SKILL.md
|
||||
|
||||
## Phase 4: Attempting to Create Anchored Comments (Failure)
|
||||
|
||||
Task: apply the review rules to a new document
|
||||
(1NmJKNavB4IBg56ZRBdS5ux9ObRVlHPnSxVmjjXCyITs — 佛法与企业管理, translator: maple).
|
||||
|
||||
### Attempt 1: comments.create without anchor
|
||||
|
||||
Result: comments appear as document-level (unanchored). The translator has no
|
||||
way to know which text each comment refers to. User rejected this approach:
|
||||
"Without anchors, there is no way to know where the translator should edit."
|
||||
|
||||
### Attempt 2: kix anchors from Docs API response
|
||||
|
||||
Searched the document body via `documents.get` for kix segment IDs.
|
||||
Result: the Docs API response does NOT expose kix anchors. Only footer IDs
|
||||
(kix.hf0, kix.list) appear. Text segments have no kix identifiers.
|
||||
|
||||
### Attempt 3: kix anchors from HTML export
|
||||
|
||||
Exported the document as HTML via Drive API `files.export_media`.
|
||||
Result: HTML export contains NO kix IDs. Since 2021, Google Docs renders
|
||||
content on `<canvas>`, so there are no DOM nodes with kix attributes in
|
||||
the exported HTML.
|
||||
|
||||
### Attempt 4: Browser-based kix extraction
|
||||
|
||||
Tried navigating to the document via browser to extract kix IDs from the
|
||||
live page DOM. Result: requires Google sign-in. Agent cannot authenticate
|
||||
to Google in a browser session.
|
||||
|
||||
### Attempt 5: Line-based anchor format (official docs)
|
||||
|
||||
The Drive API docs describe an alternative anchor format using line numbers:
|
||||
```json
|
||||
{"region": {"kind": "drive#commentRegion", "line": 1, "rev": "head"}}
|
||||
```
|
||||
Created a test comment with this format. Pending user verification on whether
|
||||
it actually anchors in the Google Docs UI.
|
||||
|
||||
### Official Position
|
||||
|
||||
The Google Drive API documentation states (as of 2025):
|
||||
|
||||
> "The anchor is saved and returned when retrieving the comment, however
|
||||
> Google Workspace editor apps treat these comments as un-anchored comments."
|
||||
|
||||
The kix.* anchor format is Google's internal, undocumented format that has
|
||||
never been reverse-engineered. A 2016 thread on the Google Apps Script
|
||||
community confirms this has been a known limitation for nearly a decade
|
||||
with no resolution.
|
||||
|
||||
## Phase 5: Workaround
|
||||
|
||||
Created `translate-files/佛法与企业管理/review-comments.dj` — a structured
|
||||
file listing each comment with:
|
||||
- The specific text it should anchor to (quoted)
|
||||
- The comment content
|
||||
|
||||
The translator can manually add these comments in the Google Docs UI by
|
||||
selecting the quoted text and inserting each comment.
|
||||
|
||||
## Key Findings
|
||||
|
||||
1. **Reading comments works perfectly** via Drive API `comments.list`
|
||||
2. **Writing anchored comments does NOT work** — the anchor is silently
|
||||
ignored by Google Docs editor apps
|
||||
3. **The only reliable way** to add anchored comments to Google Docs is
|
||||
through the browser UI (select text → Insert → Comment), which requires
|
||||
human interaction or a headless browser with Google authentication
|
||||
4. **Google Apps Script** might support anchored comments from within
|
||||
the document environment (untested — requires different auth model)
|
||||
5. The `translation-review-comment` skill remains useful for `.dj` file
|
||||
review where `patch` can be used instead of comments
|
||||
|
||||
## Files Created
|
||||
|
||||
- toolkit/skills/mpi-translation-review-comment/SKILL.md — systematic review rule checklist
|
||||
- translate-files/佛法与企业管理/review-comments.dj — manual comment reference
|
||||
Binary file not shown.
@@ -0,0 +1,7 @@
|
||||
Deepseek V4 is fast and cheap
|
||||
V4 can hardly find more issues if you ask it to reread.
|
||||
|
||||
Minimax M3 holds the most effort, but is less capable than V4
|
||||
M3 generally finds more issues if you ask it to reread.
|
||||
|
||||
GLM thinks too much before acting.
|
||||
@@ -0,0 +1,106 @@
|
||||
# MPI Bilingual Typst Template — Design Decisions
|
||||
|
||||
## Location
|
||||
|
||||
- Repo / project root: `~/documents/mpi/`
|
||||
- Template directory (contains `lib/`): `~/documents/mpi/translate-files/`
|
||||
- Template: `translate-files/lib/mpi-bilingual-template.typ`
|
||||
- Example usage: `translate-files/从物品整理到心灵整理/mindful-organizing.typ`
|
||||
|
||||
## Goal
|
||||
|
||||
A single reusable Typst template for bilingual Chinese-English 静心学堂丛书
|
||||
publications. Host `.typ` files should contain almost no setup code — only an
|
||||
`#import`, a `#show: mpi-bilingual.with(...)` call, and the article content.
|
||||
|
||||
## Font choice
|
||||
|
||||
- **Chinese:** Noto Serif CJK SC (Google Fonts)
|
||||
- **English:** Noto Serif (Google Fonts), used as a freely available substitute
|
||||
for Times New Roman.
|
||||
|
||||
## Page and paragraph formatting
|
||||
|
||||
| Setting | Value | Rationale |
|
||||
|---|---|---|
|
||||
| Paper | A4 | Standard publication size |
|
||||
| Margins | 2.5 cm all around | Matches existing documents |
|
||||
| Body size | 10.5 pt | 五号 per the formatting guide |
|
||||
| Line spacing (`leading`) | 1.5 em | 1.5× line height per the guide |
|
||||
| Paragraph spacing | 1.5 em | Equivalent to one blank line between paragraphs |
|
||||
| Alignment | justified | Matches the existing `mindful-organizing.typ` look |
|
||||
| First-line indent | 2 em | Keeps bilingual paragraphs visually distinct |
|
||||
| Heading numbering | none | Headings are written verbatim in the host file |
|
||||
| Heading spacing | `above: 2em`, `below: 1.2em` | Clear section breaks without page breaks |
|
||||
| Footnotes | 8 pt, left aligned | Per the formatting guide |
|
||||
|
||||
## Title page
|
||||
|
||||
Titles and subtitles are passed as free Typst content (`title-cn`, `title-en`,
|
||||
`subtitle-cn`, `subtitle-en`).
|
||||
|
||||
The gap between title and subtitle uses `linebreak()` inside a single paragraph
|
||||
so the spacing follows the `line-height: 1.5` rhythm instead of an arbitrary
|
||||
`v()` length. A larger `v(1.5em)` separates the Chinese block from the English
|
||||
block.
|
||||
|
||||
## Table of contents
|
||||
|
||||
The TOC shows only English headings with dotted leaders and page numbers.
|
||||
Chinese headings are displayed as compact section headers above their English
|
||||
counterparts. This mirrors the behavior of the old `helpers.typ` but lives in
|
||||
the template.
|
||||
|
||||
## Host file contract
|
||||
|
||||
A host file should look like this and nothing else:
|
||||
|
||||
```typst
|
||||
#import "../lib/mpi-bilingual-template.typ": mpi-bilingual
|
||||
|
||||
#show: mpi-bilingual.with(
|
||||
title-cn: [中文标题],
|
||||
title-en: [English Title],
|
||||
subtitle-cn: [——副标题],
|
||||
subtitle-en: [—Subtitle],
|
||||
)
|
||||
|
||||
= 一、章节标题
|
||||
= I. Chapter Title
|
||||
|
||||
中文段落……
|
||||
|
||||
English paragraph …
|
||||
```
|
||||
|
||||
Compile with the helper script:
|
||||
|
||||
```bash
|
||||
~/documents/mpi/toolkit/scripts/compile-typst.fish ./从物品整理到心灵整理/mindful-organizing.typ [output.pdf]
|
||||
```
|
||||
|
||||
Or manually from the template directory (`~/documents/mpi/translate-files/`):
|
||||
|
||||
```bash
|
||||
cd ~/documents/mpi/translate-files
|
||||
typst compile --root . ./从物品整理到心灵整理/mindful-organizing.typ
|
||||
```
|
||||
|
||||
The `--root .` is required because the import `../lib/...` would otherwise
|
||||
escape Typst's sandbox.
|
||||
|
||||
## What is intentionally not in the template
|
||||
|
||||
- No inline helpers like `#speaker` or `#quote-cn` in host files. The template
|
||||
sets global formatting only. Dialogue speakers can be marked with `**Name:**`
|
||||
when truly needed.
|
||||
- No automatic blue styling for scripture quotes, because reliably detecting
|
||||
quoted classics in plain text requires explicit markup.
|
||||
- No per-chapter footnote numbering; Typst's default continuous numbering is
|
||||
used.
|
||||
|
||||
## References
|
||||
|
||||
- `references/静心学堂丛书英文统一格式-20260211.pdf` — MPI formatting guide
|
||||
- `translate-files/从物品整理到心灵整理/mindful-organizing.typ` — example host
|
||||
file and current reference implementation
|
||||
@@ -1,20 +0,0 @@
|
||||
#!/usr/bin/env fish
|
||||
# Compile a Typst file to PDF.
|
||||
# Usage: compile-typst <path-to-file.typ> [output.pdf]
|
||||
# Output defaults to /tmp/<basename>.pdf
|
||||
# The Typst project root is set to the parent of the file's directory
|
||||
# (so imports like ../lib/... resolve correctly).
|
||||
|
||||
set src (realpath $argv[1])
|
||||
set src_dir (dirname $src)
|
||||
set root (dirname $src_dir)
|
||||
|
||||
if set -q argv[2]
|
||||
set out "$argv[2]"
|
||||
else
|
||||
set base (basename $src .typ)
|
||||
set out "/tmp/$base.pdf"
|
||||
end
|
||||
|
||||
typst compile --root $root $src $out
|
||||
echo $out
|
||||
@@ -1,14 +0,0 @@
|
||||
#!/usr/bin/env fish
|
||||
# Convert target.dj to English docx
|
||||
# Usage: dj2docx <path-to-target.dj> [output-filename]
|
||||
# Output filename defaults to /tmp/<parent-dirname>-英文.docx
|
||||
|
||||
set tgt (realpath $argv[1])
|
||||
if set -q argv[2]
|
||||
set out "$argv[2]"
|
||||
else
|
||||
set parent (basename (dirname $tgt))
|
||||
set out "/tmp/$parent-英文.docx"
|
||||
end
|
||||
pandoc $tgt -f djot -t docx -o $out
|
||||
echo $out
|
||||
@@ -1,13 +0,0 @@
|
||||
#!/usr/bin/env fish
|
||||
# Convert .docx to .dj (djot) via pandoc
|
||||
# Usage: docx2dj.fish <input.docx> [output.dj]
|
||||
# No output path → stdout
|
||||
|
||||
set docx (realpath $argv[1])
|
||||
|
||||
if test (count $argv) -ge 2
|
||||
pandoc $docx -f docx -t djot --wrap=none -o $argv[2]
|
||||
echo $argv[2]
|
||||
else
|
||||
pandoc $docx -f docx -t djot --wrap=none
|
||||
end
|
||||
@@ -1,130 +0,0 @@
|
||||
"""Generate bilingual.dj from DOCX manuscript only (no PDF).
|
||||
Source: Chinese from DOCX. Target: English from DOCX.
|
||||
"""
|
||||
import re, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DOCX = ROOT / "translate-files/佛教徒的人生态度/定稿 佛教徒的人生态度 善鑫慧炬照禅道靖妙一观轩慈德20260527.docx"
|
||||
OUT_DIR = ROOT / "translate-files/佛教徒的人生态度"
|
||||
|
||||
def has_cjk(s):
|
||||
return any('\u4e00' <= c <= '\u9fff' for c in s)
|
||||
|
||||
def pandoc(path):
|
||||
r = subprocess.run(['pandoc', path, '-f', 'docx', '-t', 'plain', '--wrap=none'],
|
||||
capture_output=True, text=True)
|
||||
return r.stdout
|
||||
|
||||
def extract_pairs(text):
|
||||
"""Return [(cn_para, en_para), ...] from body onwards."""
|
||||
lines = text.split('\n')
|
||||
body_start = None
|
||||
for i, l in enumerate(lines):
|
||||
if '生活在这个世间' in l:
|
||||
body_start = i
|
||||
break
|
||||
pairs = []
|
||||
i = body_start
|
||||
while i < len(lines):
|
||||
cn = lines[i].strip()
|
||||
if not cn or not has_cjk(cn):
|
||||
i += 1
|
||||
continue
|
||||
en = ''
|
||||
if i + 2 < len(lines) and lines[i+1].strip() == '':
|
||||
ec = lines[i+2].strip()
|
||||
if ec and not has_cjk(ec):
|
||||
en = ec
|
||||
i += 3
|
||||
else:
|
||||
i += 1
|
||||
else:
|
||||
i += 1
|
||||
continue
|
||||
pairs.append((cn, en))
|
||||
return pairs
|
||||
|
||||
SANSKRIT = [
|
||||
'bodhisattva', 'bodhicitta', 'samsara', 'Dharma', 'karma',
|
||||
'nirvana', 'Sangha', 'sutra', 'Mahayana', 'Sravaka',
|
||||
'Vinaya', 'Lamrim', 'Ksitigarbha', 'Samantabhadra',
|
||||
'Chan', 'Arhatship', 'Theravada',
|
||||
]
|
||||
|
||||
def apply_fixes(en_text, italicized):
|
||||
"""Apply typesetting fixes to English text."""
|
||||
# Fix: "2.How" → "2. How"
|
||||
en_text = re.sub(r'(\d)\.([A-Z][a-z])', r'\1. \2', en_text)
|
||||
# Fix: "said,"When → "said, "When
|
||||
en_text = re.sub(r'(said|says),"', r'\1, "', en_text)
|
||||
# Italicize Sanskrit on first occurrence
|
||||
for term in SANSKRIT:
|
||||
if term not in italicized:
|
||||
pattern = re.compile(r'\b' + re.escape(term) + r'\b')
|
||||
m = pattern.search(en_text)
|
||||
if m:
|
||||
s, e = m.start(), m.end()
|
||||
en_text = en_text[:s] + '*' + en_text[s:e] + '*' + en_text[e:]
|
||||
italicized.add(term)
|
||||
return en_text
|
||||
|
||||
def generate(pairs, out_path):
|
||||
italicized = set()
|
||||
lines = []
|
||||
|
||||
# Title
|
||||
lines.append('# 佛教徒的人生态度')
|
||||
lines.append('# The Life Attitudes of Buddhists')
|
||||
lines.append('')
|
||||
lines.append('------2014年秋讲于第九届菩提静修营')
|
||||
lines.append('---Lecture Given at the 9th Bodhi Meditation Retreat, 2014')
|
||||
lines.append('')
|
||||
lines.append(' 济群法师 ')
|
||||
lines.append('Master Jiqun')
|
||||
lines.append('')
|
||||
|
||||
# TOC
|
||||
lines.append('- 一、消极还是积极')
|
||||
lines.append('- 二、悲观还是乐观')
|
||||
lines.append('- 三、禁欲还是纵欲')
|
||||
lines.append('- 四、重生还是重死')
|
||||
lines.append('- 五、自利还是利他')
|
||||
lines.append('- 六、出世还是入世')
|
||||
lines.append('- 七、无情还是多情')
|
||||
lines.append('- 八、随缘还是进取')
|
||||
lines.append('- 九、结束语')
|
||||
lines.append('')
|
||||
lines.append('- I. Passive or Proactive')
|
||||
lines.append('- II. Pessimism or Optimism')
|
||||
lines.append('- III. Abstinence or Indulgence')
|
||||
lines.append('- IV. Focus on Life or on Death')
|
||||
lines.append('- V. Benefit Oneself or Benefit Others')
|
||||
lines.append('- VI. Transcending the World or Engaging with the World')
|
||||
lines.append('- VII. To Love or Not to Love')
|
||||
lines.append('- VIII. Adapting to Conditions or Striving for Progress')
|
||||
lines.append('- IX. Conclusion')
|
||||
lines.append('')
|
||||
|
||||
# Body
|
||||
for cn, en in pairs:
|
||||
en_fixed = apply_fixes(en, italicized)
|
||||
lines.append(cn)
|
||||
lines.append(en_fixed)
|
||||
lines.append('')
|
||||
|
||||
with open(out_path, 'w') as f:
|
||||
f.write('\n'.join(lines))
|
||||
|
||||
print(f"Written: {out_path}")
|
||||
print(f" Paragraphs: {len(pairs)}")
|
||||
print(f" Sanskrit italicized: {sorted(italicized)}")
|
||||
|
||||
if __name__ == '__main__':
|
||||
print("Extracting DOCX...")
|
||||
text = pandoc(DOCX)
|
||||
pairs = extract_pairs(text)
|
||||
print(f" Pairs: {len(pairs)}")
|
||||
|
||||
out = OUT_DIR / "bilingual.dj"
|
||||
generate(pairs, out)
|
||||
@@ -1,262 +0,0 @@
|
||||
"""
|
||||
Generate bilingual.dj for 佛教徒的人生态度.
|
||||
Source: Chinese from DOCX manuscript.
|
||||
Target: English from PDF typeset.
|
||||
|
||||
Strategy: find each DOCX English paragraph in PDF body, extract
|
||||
the PDF text region for that paragraph using position boundaries.
|
||||
"""
|
||||
import re, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DOCX = ROOT / "translate-files/佛教徒的人生态度/定稿 佛教徒的人生态度 善鑫慧炬照禅道靖妙一观轩慈德20260527.docx"
|
||||
PDF = ROOT / "translate-files/佛教徒的人生态度/0607-二排-果澄-佛教徒的人生态度-一校-多人-0607.pdf"
|
||||
OUT_DIR = ROOT / "translate-files/佛教徒的人生态度"
|
||||
|
||||
|
||||
def has_cjk(s):
|
||||
return any('\u4e00' <= c <= '\u9fff' for c in s)
|
||||
|
||||
|
||||
def pandoc(path):
|
||||
r = subprocess.run(['pandoc', path, '-f', 'docx', '-t', 'plain', '--wrap=none'],
|
||||
capture_output=True, text=True)
|
||||
return r.stdout
|
||||
|
||||
|
||||
def extract_docx_pairs(text):
|
||||
"""Return [(cn_para, en_para), ...] from body onwards."""
|
||||
lines = text.split('\n')
|
||||
body_start = None
|
||||
for i, l in enumerate(lines):
|
||||
if '生活在这个世间' in l:
|
||||
body_start = i
|
||||
break
|
||||
pairs = []
|
||||
i = body_start
|
||||
while i < len(lines):
|
||||
cn = lines[i].strip()
|
||||
if not cn or not has_cjk(cn):
|
||||
i += 1
|
||||
continue
|
||||
en = ''
|
||||
if i + 2 < len(lines) and lines[i+1].strip() == '':
|
||||
ec = lines[i+2].strip()
|
||||
if ec and not has_cjk(ec):
|
||||
en = ec
|
||||
i += 3
|
||||
else:
|
||||
i += 1
|
||||
else:
|
||||
i += 1
|
||||
continue
|
||||
pairs.append((cn, en))
|
||||
return pairs
|
||||
|
||||
|
||||
def extract_pdf_body(text):
|
||||
"""Return cleaned PDF body string."""
|
||||
lines = text.split('\n')
|
||||
body_start = None
|
||||
for i, l in enumerate(lines):
|
||||
if 'iving in this world' in l.strip():
|
||||
body_start = i
|
||||
break
|
||||
slug_re = re.compile(r'佛教徒的人生态度.*indd \d+')
|
||||
hdr_re = re.compile(r'^(The Life Attitudes of Buddhists|The Mindful Peace Academy Collection)$')
|
||||
pn_re = re.compile(r'^\d{1,3}$')
|
||||
|
||||
tl = []
|
||||
for i in range(body_start, len(lines)):
|
||||
s = lines[i].strip()
|
||||
if not s or s == '\x0c':
|
||||
continue
|
||||
if slug_re.search(s) or hdr_re.match(s) or pn_re.match(s):
|
||||
continue
|
||||
tl.append(s)
|
||||
|
||||
# Join hyphenation breaks — handle consecutive breaks
|
||||
joined = []
|
||||
i = 0
|
||||
while i < len(tl):
|
||||
line = tl[i].rstrip()
|
||||
if line.endswith('-') and i + 1 < len(tl):
|
||||
n = tl[i+1].lstrip()
|
||||
if n and n[0].islower():
|
||||
merged = line[:-1] + n
|
||||
# Check if MORE consecutive breaks follow
|
||||
j = i + 2
|
||||
while j < len(tl) and merged.rstrip().endswith('-'):
|
||||
nn = tl[j].lstrip()
|
||||
if nn and nn[0].islower():
|
||||
merged = merged.rstrip()[:-1] + nn
|
||||
j += 1
|
||||
else:
|
||||
break
|
||||
joined.append(merged)
|
||||
i = j
|
||||
continue
|
||||
joined.append(line)
|
||||
i += 1
|
||||
|
||||
body = ' '.join(joined)
|
||||
body = re.sub(r'\s+', ' ', body).strip()
|
||||
body = body.replace('L iving', 'Living')
|
||||
return body
|
||||
|
||||
|
||||
def norm(s):
|
||||
s = re.sub(r'\s+', ' ', s).strip().lower()
|
||||
s = s.replace('\u201c', '"').replace('\u201d', '"')
|
||||
s = s.replace('\u2018', "'").replace('\u2019', "'")
|
||||
return s
|
||||
|
||||
|
||||
def find_positions(pairs, pdf_body):
|
||||
"""For each DOCX English para, find start position in PDF body.
|
||||
Returns list of (start_pos or None, matched_text or None).
|
||||
"""
|
||||
positions = []
|
||||
last_pos = 0
|
||||
for cn, en in pairs:
|
||||
needle = norm(en)
|
||||
haystack = norm(pdf_body[last_pos:])
|
||||
|
||||
# Try full match
|
||||
idx = haystack.find(needle)
|
||||
if idx < 0:
|
||||
# Try first 80 chars
|
||||
key = needle[:80]
|
||||
idx = haystack.find(key)
|
||||
if idx < 0:
|
||||
# Try first 40 chars
|
||||
key = needle[:40]
|
||||
idx = haystack.find(key)
|
||||
if idx < 0:
|
||||
# Try first 25 chars
|
||||
key = needle[:25]
|
||||
idx = haystack.find(key)
|
||||
|
||||
if idx >= 0:
|
||||
pos = last_pos + idx
|
||||
positions.append(pos)
|
||||
last_pos = pos + max(len(needle), 30)
|
||||
else:
|
||||
positions.append(None)
|
||||
return positions
|
||||
|
||||
|
||||
def extract_segments(pdf_body, positions):
|
||||
"""For each position, extract the PDF text region.
|
||||
Region extends from positions[i] to positions[i+1] (or end),
|
||||
trimmed to avoid bleeding into the next paragraph.
|
||||
"""
|
||||
segments = []
|
||||
for i, pos in enumerate(positions):
|
||||
if pos is None:
|
||||
segments.append(None)
|
||||
continue
|
||||
|
||||
start = pos
|
||||
end = len(pdf_body)
|
||||
for j in range(i + 1, len(positions)):
|
||||
if positions[j] is not None:
|
||||
end = positions[j]
|
||||
break
|
||||
|
||||
raw = pdf_body[start:end].strip()
|
||||
|
||||
# Trim: if raw contains what looks like the NEXT paragraph's heading,
|
||||
# cut at the last sentence boundary before it.
|
||||
# Headings match patterns like: "I Passive", "1) Expressions", "1. The Definitions"
|
||||
heading_pattern = re.compile(
|
||||
r'\s+(?=[IVX]+\.?\s+[A-Z]' # Roman numeral chapter
|
||||
r'|\d+\)\s+[A-Z]' # 1) Sub-heading
|
||||
r'|\(\d+\)\s+[A-Z]' # (1) Sub-heading
|
||||
r'|\d+\.\s+[A-Z][a-z]+.*?(?:Passive|Pessimism|Abstinence|Focus|Benefit|Transcending|Love|Adapting|Conclusion|Desire|Being|What|How|The|Buddhism|Set|Free|A Middle)' # Numbered heading
|
||||
r')'
|
||||
)
|
||||
m = heading_pattern.search(raw)
|
||||
if m:
|
||||
# Cut before this heading
|
||||
raw = raw[:m.start()].strip()
|
||||
|
||||
segments.append(raw)
|
||||
return segments
|
||||
|
||||
|
||||
def generate(pairs, segments, out_path):
|
||||
lines = []
|
||||
|
||||
# Title
|
||||
lines.append('# 佛教徒的人生态度')
|
||||
lines.append('# The Life Attitudes of Buddhists')
|
||||
lines.append('')
|
||||
lines.append('------2014年秋讲于第九届菩提静修营')
|
||||
lines.append('---Lecture Given at the 9th Bodhi Meditation Retreat, 2014')
|
||||
lines.append('')
|
||||
lines.append(' 济群法师 ')
|
||||
lines.append('Master Jiqun')
|
||||
lines.append('')
|
||||
|
||||
# TOC from DOCX
|
||||
lines.append('- 一、消极还是积极')
|
||||
lines.append('- 二、悲观还是乐观')
|
||||
lines.append('- 三、禁欲还是纵欲')
|
||||
lines.append('- 四、重生还是重死')
|
||||
lines.append('- 五、自利还是利他')
|
||||
lines.append('- 六、出世还是入世')
|
||||
lines.append('- 七、无情还是多情')
|
||||
lines.append('- 八、随缘还是进取')
|
||||
lines.append('- 九、结束语')
|
||||
lines.append('')
|
||||
lines.append('- I. Passive or Proactive')
|
||||
lines.append('- II. Pessimism or Optimism')
|
||||
lines.append('- III. Abstinence or Indulgence')
|
||||
lines.append('- IV. Focus on Life or on Death')
|
||||
lines.append('- V. Benefit Oneself or Benefit Others')
|
||||
lines.append('- VI. Transcending the World or Engaging with the World')
|
||||
lines.append('- VII. To Love or Not to Love')
|
||||
lines.append('- VIII. Adapting to Conditions or Striving for Progress')
|
||||
lines.append('- IX. Conclusion')
|
||||
lines.append('')
|
||||
|
||||
# Body
|
||||
for (cn, en), seg in zip(pairs, segments):
|
||||
target = seg if seg else en
|
||||
lines.append(cn)
|
||||
lines.append(target)
|
||||
lines.append('')
|
||||
|
||||
with open(out_path, 'w') as f:
|
||||
f.write('\n'.join(lines))
|
||||
|
||||
matched = sum(1 for s in segments if s is not None)
|
||||
print(f"Written: {out_path}")
|
||||
print(f" Paragraphs: {len(pairs)}, matched from PDF: {matched}, fallback to DOCX: {len(pairs) - matched}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
print("Extracting DOCX...")
|
||||
docx_text = pandoc(DOCX)
|
||||
pairs = extract_docx_pairs(docx_text)
|
||||
print(f" Pairs: {len(pairs)}")
|
||||
|
||||
print("Extracting PDF...")
|
||||
r = subprocess.run(['pdftotext', '-layout', PDF, '/tmp/_bilingual_pdf.txt'], check=True)
|
||||
with open('/tmp/_bilingual_pdf.txt') as f:
|
||||
pdf_raw = f.read()
|
||||
pdf_body = extract_pdf_body(pdf_raw)
|
||||
print(f" PDF body: {len(pdf_body)} chars")
|
||||
|
||||
print("Finding positions...")
|
||||
positions = find_positions(pairs, pdf_body)
|
||||
found = sum(1 for p in positions if p is not None)
|
||||
print(f" Found: {found}/{len(pairs)}")
|
||||
|
||||
print("Extracting segments...")
|
||||
segments = extract_segments(pdf_body, positions)
|
||||
|
||||
out = OUT_DIR / "bilingual.dj"
|
||||
generate(pairs, segments, out)
|
||||
@@ -1,193 +0,0 @@
|
||||
"""Extract source.dj and bilingual.dj from .docx.md file.
|
||||
|
||||
Handles:
|
||||
- CN/EN paragraph pairs (CN line → EN line)
|
||||
- Headings with merged CN+EN on same line (pandoc artifact: CN**EN)
|
||||
- TOC with markdown links [CN text](#anchor)
|
||||
- {#anchor} pandoc heading anchors
|
||||
- Markdown heading cleanup
|
||||
"""
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
SRC = ROOT / "translate-files/佛法与企业管理/副本59 佛法与企业管理-maple 初翻.docx.md"
|
||||
OUT = ROOT / "translate-files/佛法与企业管理"
|
||||
|
||||
def has_cjk(s):
|
||||
return any('\u4e00' <= c <= '\u9fff' for c in s)
|
||||
|
||||
def strip_anchors(s):
|
||||
"""Remove {#anchor} and markdown links [text](#anchor) — keep text."""
|
||||
s = re.sub(r'\{#[^}]*\}', '', s) # {#anchor}
|
||||
s = re.sub(r'\[([^\]]*)\]\([^)]*\)', r'\1', s) # [text](#link) → text
|
||||
return s
|
||||
|
||||
def split_cnen(line):
|
||||
"""Split merged CN+EN line. First strips anchors, then finds CJK→EN boundary."""
|
||||
s = strip_anchors(line).strip()
|
||||
if not has_cjk(s):
|
||||
return ('', s)
|
||||
# Find where CJK ends and ASCII English begins
|
||||
# Pattern: CJK, optional ws, optional *, optional ws, then English letter
|
||||
m = re.search(r'[\u4e00-\u9fff\u3000-\u303f\uff00-\uffef]\s*\**\s*([A-Za-z])', s)
|
||||
if m:
|
||||
split_at = m.start(1)
|
||||
cn = s[:split_at].rstrip('* ').strip()
|
||||
en = s[split_at:].strip()
|
||||
if en and not has_cjk(en):
|
||||
return (cn, en)
|
||||
return (s, '')
|
||||
|
||||
def clean_cn(s):
|
||||
"""Clean CN heading/paragraph."""
|
||||
s = re.sub(r'^\d+\.\s*', '', s) # leading number (e.g. "1. ")
|
||||
s = re.sub(r'^#+\s*\**', '', s) # heading markers: # **
|
||||
s = re.sub(r'\**\s*$', '', s) # trailing **
|
||||
s = re.sub(r'\t\d+$', '', s) # trailing page number
|
||||
s = s.strip()
|
||||
return s
|
||||
|
||||
def clean_en(s):
|
||||
"""Clean EN line."""
|
||||
s = re.sub(r'^\d+[、,.]\s*', '', s) # leading number/separator
|
||||
s = re.sub(r'\*+$', '', s) # trailing orphaned italic * (from split)
|
||||
s = s.strip()
|
||||
return s
|
||||
|
||||
def is_toc_line(line, cn_raw):
|
||||
"""True if this looks like a TOC entry (markdown link or tab+page-number)."""
|
||||
if re.search(r'\[.*\]\(.*\)', line):
|
||||
return True
|
||||
if re.search(r'\t\d+', cn_raw):
|
||||
return True
|
||||
return False
|
||||
|
||||
def parse_doc(text):
|
||||
"""Parse into list of (cn, en) pairs."""
|
||||
lines = text.split('\n')
|
||||
pairs = []
|
||||
i = 0
|
||||
|
||||
while i < len(lines):
|
||||
line = lines[i].strip()
|
||||
if not line:
|
||||
i += 1
|
||||
continue
|
||||
|
||||
stripped = strip_anchors(line)
|
||||
has_cn = has_cjk(stripped)
|
||||
|
||||
if has_cn:
|
||||
cn_raw, en_raw = split_cnen(line)
|
||||
|
||||
if en_raw:
|
||||
# Merged CN+EN on same line
|
||||
pairs.append((clean_cn(cn_raw), clean_en(en_raw)))
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Look ahead for EN
|
||||
nxt = lines[i+1].strip() if i+1 < len(lines) else ''
|
||||
nnxt = lines[i+2].strip() if i+2 < len(lines) else ''
|
||||
|
||||
if nxt and not has_cjk(nxt) and not is_toc_line(line, cn_raw):
|
||||
# Standard: CN → EN
|
||||
pairs.append((clean_cn(cn_raw), clean_en(nxt)))
|
||||
i += 2
|
||||
elif not nxt and nnxt and not has_cjk(nnxt) and not is_toc_line(line, cn_raw):
|
||||
# CN → blank → EN (heading pattern)
|
||||
pairs.append((clean_cn(cn_raw), clean_en(nnxt)))
|
||||
i += 3
|
||||
else:
|
||||
# Solo CN (TOC entry, orphan, or heading)
|
||||
pairs.append((clean_cn(cn_raw), ''))
|
||||
i += 1
|
||||
else:
|
||||
# Pure EN — TOC entry, pair with first unpaired CN TOC entry
|
||||
en = clean_en(line)
|
||||
for j in range(len(pairs)):
|
||||
if not pairs[j][1] and has_cjk(pairs[j][0]):
|
||||
pairs[j] = (pairs[j][0], en)
|
||||
break
|
||||
else:
|
||||
pairs.append(('', en))
|
||||
i += 1
|
||||
|
||||
return pairs
|
||||
|
||||
def classify(pairs):
|
||||
"""Classify each pair as title, subtitle, toc, heading, or para."""
|
||||
result = []
|
||||
# Pairs 0-1: title and subtitle
|
||||
result.append(('title', 0))
|
||||
result.append(('subtitle', 1))
|
||||
# Pairs 2-9: TOC (一、 through 八、)
|
||||
for i in range(2, min(10, len(pairs))):
|
||||
result.append(('toc', i))
|
||||
# Remaining: heuristics
|
||||
for i in range(10, len(pairs)):
|
||||
cn = pairs[i][0]
|
||||
if re.match(r'^[一二三四五六七八九十]、', cn):
|
||||
result.append(('heading', i))
|
||||
elif re.match(r'^\d+\\?\.\s', cn):
|
||||
result.append(('subheading', i))
|
||||
else:
|
||||
result.append(('para', i))
|
||||
return result
|
||||
|
||||
def write_bilingual(pairs, out_path):
|
||||
types = classify(pairs)
|
||||
lines = []
|
||||
# Title
|
||||
lines.append('# ' + pairs[0][0])
|
||||
lines.append('# ' + pairs[0][1])
|
||||
lines.append('')
|
||||
# Subtitle
|
||||
lines.append('---' + pairs[1][0])
|
||||
lines.append('---' + pairs[1][1])
|
||||
lines.append('')
|
||||
# Author
|
||||
lines.append('济群法师')
|
||||
lines.append('Master Jiqun')
|
||||
lines.append('')
|
||||
# TOC — CN block then EN block (not interleaved)
|
||||
for typ, idx in types:
|
||||
if typ == 'toc':
|
||||
lines.append('- ' + pairs[idx][0])
|
||||
lines.append('')
|
||||
for typ, idx in types:
|
||||
if typ == 'toc':
|
||||
lines.append('- ' + pairs[idx][1])
|
||||
lines.append('')
|
||||
# Body
|
||||
for typ, idx in types:
|
||||
if typ in ('title', 'subtitle', 'toc'):
|
||||
continue
|
||||
cn, en = pairs[idx]
|
||||
if cn:
|
||||
lines.append(cn)
|
||||
if en:
|
||||
lines.append(en)
|
||||
if cn or en:
|
||||
lines.append('')
|
||||
with open(out_path, 'w') as f:
|
||||
f.write('\n'.join(lines))
|
||||
print(f"bilingual.dj: {len(pairs)} pairs -> {out_path}")
|
||||
|
||||
def write_source(pairs, out_path):
|
||||
lines = [cn for cn, en in pairs if cn]
|
||||
with open(out_path, 'w') as f:
|
||||
f.write('\n'.join(lines) + '\n')
|
||||
print(f"source.dj: {len(lines)} CN lines -> {out_path}")
|
||||
|
||||
if __name__ == '__main__':
|
||||
text = SRC.read_text()
|
||||
pairs = parse_doc(text)
|
||||
print(f"Parsed {len(pairs)} pairs from .docx.md")
|
||||
solo_cn = sum(1 for cn, en in pairs if cn and not en)
|
||||
solo_en = sum(1 for cn, en in pairs if en and not cn)
|
||||
both = sum(1 for cn, en in pairs if cn and en)
|
||||
print(f" Both: {both}, CN-only: {solo_cn}, EN-only: {solo_en}")
|
||||
write_source(pairs, OUT / "source.dj")
|
||||
write_bilingual(pairs, OUT / "bilingual.dj")
|
||||
@@ -1,183 +0,0 @@
|
||||
"""Generate bilingual.dj from DOCX for 「生命也可以被设计的」.
|
||||
One-pass approach: walk interleaved paragraphs, handle multi-CN sequences.
|
||||
"""
|
||||
import re, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
DOCX = ROOT / "translate-files/生命也可以被设计的/中英文定稿-260324-生命也是可以被设计的-妙一宽山静雅初翻 慈鎏妙一审议 宽山定稿.docx"
|
||||
OUT_DIR = ROOT / "translate-files/生命也可以被设计的"
|
||||
|
||||
def has_cjk(s):
|
||||
return any('\u4e00' <= c <= '\u9fff' for c in s)
|
||||
|
||||
def pandoc(path):
|
||||
r = subprocess.run(['pandoc', path, '-f', 'docx', '-t', 'plain', '--wrap=none'],
|
||||
capture_output=True, text=True)
|
||||
return r.stdout
|
||||
|
||||
def split_toc_line(line):
|
||||
s = line.strip()
|
||||
s = re.sub(r'\s+\d+\s*$', '', s)
|
||||
m = re.match(r'^(.+[\u4e00-\u9fff\u3000-\u303f\uff00-\uffef）\)])\s+([A-Z].+)$', s)
|
||||
if m:
|
||||
return m.group(1).strip(), m.group(2).strip()
|
||||
return None, None
|
||||
|
||||
def extract_toc_entries(text):
|
||||
lines = text.split('\n')
|
||||
toc_start = None
|
||||
toc_end = None
|
||||
for i, l in enumerate(lines):
|
||||
s = l.strip()
|
||||
if s.startswith('一、') and ('EDUCATION' in s or 'NURTURING' in s):
|
||||
if toc_start is None:
|
||||
toc_start = i
|
||||
if toc_start is not None and s and has_cjk(s) and re.search(r'\d+$', s):
|
||||
toc_end = i
|
||||
elif toc_start is not None and toc_end is not None and s and not re.search(r'\d+$', s) and has_cjk(s):
|
||||
break
|
||||
|
||||
cn_entries = []
|
||||
en_entries = []
|
||||
for i in range(toc_start, toc_end + 1):
|
||||
cn, en = split_toc_line(lines[i])
|
||||
if cn and en:
|
||||
cn_entries.append(cn)
|
||||
en_entries.append(en)
|
||||
return cn_entries, en_entries
|
||||
|
||||
def extract_body_pairs(text):
|
||||
"""One-pass: walk interleaved paras, joining consecutive same-language lines."""
|
||||
lines = text.split('\n')
|
||||
|
||||
# Find body start
|
||||
body_start = None
|
||||
for i, l in enumerate(lines):
|
||||
if '现在是一个浮躁的时代' in l:
|
||||
body_start = i
|
||||
break
|
||||
|
||||
# Extract non-blank paragraphs with language tags
|
||||
tagged = []
|
||||
for l in lines[body_start:]:
|
||||
s = l.strip()
|
||||
if s:
|
||||
tagged.append(('cn' if has_cjk(s) else 'en', s))
|
||||
|
||||
# Accumulate consecutive same-language paragraphs (page-break splits only, not headings)
|
||||
merged = []
|
||||
for lang, text in tagged:
|
||||
# Heading-like patterns that should not be merged
|
||||
prev_is_heading = (merged and merged[-1][0] == lang
|
||||
and bool(re.match(r'^[\dIVX]+[\.\s]', merged[-1][1].strip())
|
||||
and len(merged[-1][1].strip()) < 60))
|
||||
if (merged and merged[-1][0] == lang
|
||||
and not prev_is_heading
|
||||
and len(merged[-1][1]) > 30
|
||||
and not re.search(r'[。！？：）\u201d\u2019\uff0c\uff0e\.!\?]$', merged[-1][1])):
|
||||
# Long previous line, doesn't end naturally → page-break split, join
|
||||
merged[-1] = (lang, merged[-1][1].rstrip() + text.lstrip())
|
||||
else:
|
||||
merged.append((lang, text))
|
||||
|
||||
# Build pairs: group consecutive same-language items into blocks, then zip
|
||||
blocks = []
|
||||
for lang, text in merged:
|
||||
if blocks and blocks[-1][0] == lang:
|
||||
blocks[-1][1].append(text)
|
||||
else:
|
||||
blocks.append((lang, [text]))
|
||||
|
||||
pairs = []
|
||||
i = 0
|
||||
while i < len(blocks):
|
||||
if blocks[i][0] == 'cn':
|
||||
cn_block = blocks[i][1]
|
||||
# Find next EN block
|
||||
if i + 1 < len(blocks) and blocks[i+1][0] == 'en':
|
||||
en_block = blocks[i+1][1]
|
||||
n = min(len(cn_block), len(en_block))
|
||||
for j in range(n):
|
||||
pairs.append((cn_block[j], en_block[j]))
|
||||
if len(cn_block) != len(en_block):
|
||||
print(f" WARNING: block mismatch CN={len(cn_block)} EN={len(en_block)} at CN[{j}]: {cn_block[j][:60]}...")
|
||||
i += 2
|
||||
else:
|
||||
print(f" WARNING: CN block without EN block: {cn_block[0][:60]}...")
|
||||
i += 1
|
||||
else:
|
||||
print(f" WARNING: orphan EN block: {blocks[i][1][0][:60]}...")
|
||||
i += 1
|
||||
|
||||
return pairs
|
||||
|
||||
SANSKRIT = [
|
||||
'bodhisattva', 'bodhicitta', 'samsara', 'Dharma', 'karma',
|
||||
'nirvana', 'Sangha', 'sutra', 'Mahayana', 'Sravaka',
|
||||
'Vinaya', 'Lamrim', 'Ksitigarbha', 'Samantabhadra',
|
||||
'Chan', 'Arhatship', 'Theravada', 'buddha', 'Buddha',
|
||||
'buddhas', 'Buddhas', 'Bodhisattva', 'Bodhisattvas',
|
||||
]
|
||||
|
||||
def apply_fixes(en_text, italicized):
|
||||
en_text = re.sub(r'(\d)\.([A-Z][a-z])', r'\1. \2', en_text)
|
||||
en_text = re.sub(r'(said|says),\"', r'\1, "', en_text)
|
||||
en_text = re.sub(r'\.([A-Z][a-z])', r'. \1', en_text)
|
||||
for term in SANSKRIT:
|
||||
if term not in italicized:
|
||||
pattern = re.compile(r'\b' + re.escape(term) + r'\b')
|
||||
m = pattern.search(en_text)
|
||||
if m:
|
||||
s, e = m.start(), m.end()
|
||||
en_text = en_text[:s] + '*' + en_text[s:e] + '*' + en_text[e:]
|
||||
italicized.add(term)
|
||||
return en_text
|
||||
|
||||
def generate(toc_cn, toc_en, pairs, out_path):
|
||||
italicized = set()
|
||||
lines = []
|
||||
|
||||
lines.append('# 生命也是可以被设计的')
|
||||
lines.append('# Life Can Also Be Designed')
|
||||
lines.append('')
|
||||
lines.append('济群法师 2025年冬为母爱书院开示')
|
||||
lines.append('A teaching given by the Master Jiqun in the winter of 2025 at Amrita Retreat Center for Motherly Love Academy')
|
||||
lines.append('')
|
||||
|
||||
for e in toc_cn:
|
||||
lines.append(f'- {e}')
|
||||
lines.append('')
|
||||
for e in toc_en:
|
||||
lines.append(f'- {e}')
|
||||
lines.append('')
|
||||
|
||||
for cn, en in pairs:
|
||||
en_fixed = apply_fixes(en, italicized)
|
||||
lines.append(cn)
|
||||
lines.append(en_fixed)
|
||||
lines.append('')
|
||||
|
||||
with open(out_path, 'w') as f:
|
||||
f.write('\n'.join(lines))
|
||||
|
||||
print(f"Written: {out_path}")
|
||||
print(f" TOC entries: {len(toc_cn)}")
|
||||
print(f" Body pairs: {len(pairs)}")
|
||||
print(f" Sanskrit italicized: {sorted(italicized)}")
|
||||
|
||||
if __name__ == '__main__':
|
||||
print("Extracting DOCX...")
|
||||
text = pandoc(DOCX)
|
||||
|
||||
print("Extracting TOC...")
|
||||
toc_cn, toc_en = extract_toc_entries(text)
|
||||
for cn, en in zip(toc_cn, toc_en):
|
||||
print(f" {cn} → {en}")
|
||||
|
||||
print("Extracting body...")
|
||||
pairs = extract_body_pairs(text)
|
||||
print(f" Pairs: {len(pairs)}")
|
||||
|
||||
out = OUT_DIR / "bilingual.dj"
|
||||
generate(toc_cn, toc_en, pairs, out)
|
||||
@@ -1,60 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a bilingual .dj file from source (Chinese) and target (English) .dj files.
|
||||
|
||||
Usage:
|
||||
gen-bilingual.py source.dj target.dj > bilingual.dj
|
||||
|
||||
Output format: source line, target line, blank line, repeated. Paragraph breaks
|
||||
are preserved: blank lines in the input produce blank lines in the output.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 3:
|
||||
print(__doc__, file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
src_path = Path(sys.argv[1])
|
||||
tgt_path = Path(sys.argv[2])
|
||||
|
||||
if not src_path.exists():
|
||||
print(f"Source file not found: {src_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
if not tgt_path.exists():
|
||||
print(f"Target file not found: {tgt_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
src_lines = src_path.read_text(encoding="utf-8").splitlines()
|
||||
tgt_lines = tgt_path.read_text(encoding="utf-8").splitlines()
|
||||
|
||||
if len(src_lines) != len(tgt_lines):
|
||||
print(
|
||||
f"Line count mismatch: source={len(src_lines)} target={len(tgt_lines)}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
out = []
|
||||
for s, t in zip(src_lines, tgt_lines):
|
||||
if s == "":
|
||||
out.append("")
|
||||
else:
|
||||
out.append(s)
|
||||
out.append(t)
|
||||
out.append("")
|
||||
|
||||
# Ensure the output always ends with a single trailing blank line to match
|
||||
# the project convention: source, target, blank, source, target, blank...
|
||||
if out and out[-1] != "":
|
||||
out.append("")
|
||||
|
||||
sys.stdout.write("\n".join(out))
|
||||
if out:
|
||||
sys.stdout.write("\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,169 +0,0 @@
|
||||
"""Compare manuscript (DOCX English body) against typeset (PDF English body).
|
||||
Usage: python3 scripts/proofread-pdf.py <docx_path> <pdf_path>
|
||||
Output: sentences from DOCX not found in PDF, and word-level changes within matched sentences.
|
||||
"""
|
||||
import re, sys, subprocess
|
||||
|
||||
|
||||
def extract_docx_en(path):
|
||||
with open(path) as f:
|
||||
lines = f.readlines()
|
||||
|
||||
body_start = None
|
||||
for i, line in enumerate(lines):
|
||||
if '生活在这个世间' in line:
|
||||
body_start = i
|
||||
break
|
||||
if body_start is None:
|
||||
sys.exit("Could not find body start in DOCX")
|
||||
|
||||
docx_en = []
|
||||
skip_next = 0
|
||||
for i in range(body_start, len(lines)):
|
||||
if skip_next > 0:
|
||||
skip_next -= 1
|
||||
continue
|
||||
line = lines[i].strip()
|
||||
if not line:
|
||||
continue
|
||||
has_cjk = any('\u4e00' <= c <= '\u9fff' for c in line)
|
||||
if has_cjk:
|
||||
if i + 1 < len(lines) and lines[i+1].strip() == '':
|
||||
if i + 2 < len(lines):
|
||||
en_line = lines[i+2].strip()
|
||||
if en_line and not any('\u4e00' <= c <= '\u9fff' for c in en_line):
|
||||
docx_en.append(en_line)
|
||||
skip_next = 2
|
||||
else:
|
||||
docx_en.append(line)
|
||||
|
||||
# Split into sentences, filter out headings
|
||||
text = ' '.join(docx_en)
|
||||
sentences = re.split(r'(?<=[.!?"”])\s+', text)
|
||||
return [(s.strip(), len(s.strip())) for s in sentences if len(s.strip()) >= 20]
|
||||
|
||||
|
||||
def extract_pdf_en(path):
|
||||
with open(path) as f:
|
||||
lines = f.readlines()
|
||||
|
||||
body_start = None
|
||||
for i, line in enumerate(lines):
|
||||
if 'iving in this world' in line:
|
||||
body_start = i
|
||||
break
|
||||
if body_start is None:
|
||||
sys.exit("Could not find body start in PDF")
|
||||
|
||||
slug_re = re.compile(r'佛教徒的人生态度.*indd \d+')
|
||||
header_re = re.compile(r'^(The Life Attitudes of Buddhists|The Mindful Peace Academy Collection)$')
|
||||
page_num_re = re.compile(r'^\d{1,3}$')
|
||||
|
||||
text_lines = []
|
||||
for i in range(body_start, len(lines)):
|
||||
s = lines[i].strip()
|
||||
if not s or s == '\x0c':
|
||||
continue
|
||||
if slug_re.search(s) or header_re.match(s) or page_num_re.match(s):
|
||||
continue
|
||||
text_lines.append(s)
|
||||
|
||||
# Join hyphenated breaks
|
||||
joined = []
|
||||
i = 0
|
||||
while i < len(text_lines):
|
||||
line = text_lines[i]
|
||||
if line.rstrip().endswith('-') and i + 1 < len(text_lines):
|
||||
n = text_lines[i+1].lstrip()
|
||||
if n and n[0].islower():
|
||||
joined.append(line.rstrip()[:-1] + n)
|
||||
i += 2
|
||||
continue
|
||||
joined.append(line)
|
||||
i += 1
|
||||
|
||||
body = ' '.join(joined)
|
||||
body = re.sub(r'\s+', ' ', body).strip()
|
||||
body = body.replace('L iving', 'Living')
|
||||
return body
|
||||
|
||||
|
||||
def normalize_for_search(s):
|
||||
"""Normalize text for fuzzy matching."""
|
||||
s = re.sub(r'\s+', ' ', s).strip().lower()
|
||||
# Normalize quotes
|
||||
s = s.replace('\u201c', '"').replace('\u201d', '"')
|
||||
s = s.replace('\u2018', "'").replace('\u2019', "'")
|
||||
return s
|
||||
|
||||
|
||||
def find_sentence_in_pdf(sentence, pdf_body):
|
||||
"""Try to locate sentence in PDF body. Returns (found, matched_text)."""
|
||||
s_norm = normalize_for_search(sentence)
|
||||
# Try full sentence
|
||||
if s_norm in pdf_body.lower():
|
||||
return True, sentence
|
||||
# Try first 60 chars
|
||||
key = s_norm[:60]
|
||||
if key in pdf_body.lower():
|
||||
return True, sentence
|
||||
# Try first 30 chars
|
||||
key = s_norm[:30]
|
||||
if key in pdf_body.lower():
|
||||
return True, sentence
|
||||
return False, None
|
||||
|
||||
|
||||
def find_word_diff(docx_sentence, pdf_sentence):
|
||||
"""Find word-level differences between two matched sentences."""
|
||||
if not pdf_sentence:
|
||||
return []
|
||||
dw = re.findall(r'\S+', docx_sentence)
|
||||
pw = re.findall(r'\S+', pdf_sentence)
|
||||
diffs = []
|
||||
for dwi, pwi in zip(dw, pw):
|
||||
if dwi.lower() != pwi.lower():
|
||||
diffs.append((dwi, pwi))
|
||||
if len(dw) != len(pw):
|
||||
diffs.append((f"[{len(dw)} words]", f"[{len(pw)} words]"))
|
||||
return diffs
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if len(sys.argv) != 3:
|
||||
sys.exit("Usage: proofread-pdf.py <docx_path> <pdf_path>")
|
||||
|
||||
docx_path, pdf_path = sys.argv[1], sys.argv[2]
|
||||
docx_txt = '/tmp/proofread_docx.txt'
|
||||
pdf_txt = '/tmp/proofread_pdf.txt'
|
||||
|
||||
subprocess.run(['pandoc', docx_path, '-f', 'docx', '-t', 'plain', '--wrap=none', '-o', docx_txt], check=True)
|
||||
subprocess.run(['pdftotext', '-layout', pdf_path, pdf_txt], check=True)
|
||||
|
||||
docx_sentences = extract_docx_en(docx_txt)
|
||||
pdf_body = extract_pdf_en(pdf_txt)
|
||||
pdf_normalized = normalize_for_search(pdf_body)
|
||||
|
||||
missing = []
|
||||
found_count = 0
|
||||
for sentence, length in docx_sentences:
|
||||
s_norm = normalize_for_search(sentence)
|
||||
if s_norm in pdf_normalized:
|
||||
found_count += 1
|
||||
elif s_norm[:60] in pdf_normalized:
|
||||
found_count += 1
|
||||
elif s_norm[:30] in pdf_normalized:
|
||||
found_count += 1
|
||||
else:
|
||||
missing.append(sentence)
|
||||
|
||||
print(f"DOCX body sentences: {len(docx_sentences)}")
|
||||
print(f"Matched in PDF: {found_count}")
|
||||
print(f"Missing: {len(missing)}")
|
||||
print()
|
||||
|
||||
if missing:
|
||||
print("=== Sentences from DOCX NOT found in PDF ===")
|
||||
for i, s in enumerate(missing):
|
||||
print(f"\n--- Missing #{i+1} ---")
|
||||
print(s[:200])
|
||||
@@ -1,22 +0,0 @@
|
||||
#!/usr/bin/env fish
|
||||
# Split combined bilingual .dj into source.dj (CN) and target.dj (EN)
|
||||
# Usage: split-bilingual.fish <combined.dj>
|
||||
# Output: source.dj and target.dj in same directory, paragraphs separated by blanks
|
||||
|
||||
set dj (realpath $argv[1])
|
||||
set dir (dirname $dj)
|
||||
|
||||
rm -f "$dir/source.dj" "$dir/target.dj"
|
||||
|
||||
for line in (cat $dj)
|
||||
if string match -qr '[\x{4e00}-\x{9fff}]' -- $line
|
||||
echo $line >> "$dir/source.dj"
|
||||
echo >> "$dir/source.dj"
|
||||
else if test -n (string trim -- $line)
|
||||
echo $line >> "$dir/target.dj"
|
||||
echo >> "$dir/target.dj"
|
||||
end
|
||||
end
|
||||
|
||||
echo "source.dj: $dir/source.dj"
|
||||
echo "target.dj: $dir/target.dj"
|
||||
@@ -1,97 +0,0 @@
|
||||
"""Extract cleaned English body from DOCX manuscript and typeset PDF.
|
||||
Usage: python3 ten-elements-c7fcd9.py <docx_path> <pdf_path>
|
||||
Output: two cleaned text files in /tmp/ for diffing.
|
||||
"""
|
||||
import re, sys, subprocess
|
||||
from pathlib import Path
|
||||
|
||||
DOCX_TXT = '/tmp/ten_elements_docx_body.txt'
|
||||
PDF_TXT = '/tmp/ten_elements_pdf_body.txt'
|
||||
|
||||
|
||||
def extract_docx_body(path):
|
||||
with open(path) as f:
|
||||
lines = f.readlines()
|
||||
for i, line in enumerate(lines):
|
||||
if 'The Dhyana Tea program team' in line:
|
||||
body_start = i
|
||||
break
|
||||
else:
|
||||
sys.exit("Could not find body start in DOCX")
|
||||
body = [l.strip() for l in lines[body_start:] if l.strip()]
|
||||
return '\n'.join(body)
|
||||
|
||||
|
||||
def extract_pdf_body(path):
|
||||
with open(path) as f:
|
||||
lines = f.readlines()
|
||||
|
||||
slug_re = re.compile(r'正念禅修十要素.*indd \d+')
|
||||
header_re = re.compile(
|
||||
r'^(The Mindful Peace Academy Collection|The Ten Key Elements of Mindfulness Meditation)$'
|
||||
)
|
||||
page_re = re.compile(r'^\d{1,3}$')
|
||||
skip_re = re.compile(
|
||||
r'^(I|II|III|IV|Three Basic Elements|The Three Key Elements of Samatha|'
|
||||
r'The Four Key Elements of Vipassana|Conclusion|Contents)$'
|
||||
)
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
if 'Dhyana Tea program team' in line.strip():
|
||||
body_start = i
|
||||
break
|
||||
else:
|
||||
sys.exit("Could not find body start in PDF")
|
||||
|
||||
raw = []
|
||||
for line in lines[body_start:]:
|
||||
s = line.strip()
|
||||
if not s or s == '\x0c':
|
||||
continue
|
||||
if slug_re.search(s) or header_re.match(s) or page_re.match(s) or skip_re.match(s):
|
||||
continue
|
||||
raw.append(s)
|
||||
|
||||
# Join hyphenated line breaks
|
||||
joined = []
|
||||
i = 0
|
||||
while i < len(raw):
|
||||
line = raw[i]
|
||||
if line.rstrip().endswith('-') and i + 1 < len(raw):
|
||||
nxt = raw[i + 1].lstrip()
|
||||
if nxt and nxt[0].islower():
|
||||
joined.append(line.rstrip()[:-1] + nxt)
|
||||
i += 2
|
||||
continue
|
||||
joined.append(line)
|
||||
i += 1
|
||||
|
||||
body = ' '.join(joined)
|
||||
body = re.sub(r'\s+', ' ', body).strip()
|
||||
body = body.replace('L iving', 'Living')
|
||||
body = re.sub(r'T\s+he\b', 'The', body)
|
||||
return body
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
if len(sys.argv) != 3:
|
||||
sys.exit(f"Usage: {Path(__file__).name} <docx_path> <pdf_path>")
|
||||
|
||||
docx_path, pdf_path = sys.argv[1], sys.argv[2]
|
||||
|
||||
subprocess.run(
|
||||
['pandoc', docx_path, '-f', 'docx', '-t', 'plain', '--wrap=none',
|
||||
'-o', '/tmp/_docx_raw.txt'], check=True
|
||||
)
|
||||
subprocess.run(
|
||||
['pdftotext', '-layout', pdf_path, '/tmp/_pdf_raw.txt'], check=True
|
||||
)
|
||||
|
||||
docx_body = extract_docx_body('/tmp/_docx_raw.txt')
|
||||
pdf_body = extract_pdf_body('/tmp/_pdf_raw.txt')
|
||||
|
||||
Path(DOCX_TXT).write_text(docx_body)
|
||||
Path(PDF_TXT).write_text(pdf_body)
|
||||
|
||||
print(f"DOCX body → {DOCX_TXT} ({len(docx_body)} chars)")
|
||||
print(f"PDF body → {PDF_TXT} ({len(pdf_body)} chars)")
|
||||
@@ -1,85 +0,0 @@
|
||||
---
|
||||
name: chinese-text-normalize
|
||||
description: Normalize Chinese markdown files — remove extraneous mid-sentence line breaks from fixed-width exports while preserving TOC structures, section headers, and intentional paragraph breaks.
|
||||
---
|
||||
|
||||
When Chinese text has hard line breaks at a fixed width (~20-25 chars) — common in PDF exports, OCR output, or poorly-converted documents — use this skill to join them into flowing paragraphs.
|
||||
|
||||
## Triggers
|
||||
|
||||
- User asks to "fix line breaks" or "remove extraneous breaks" in Chinese text
|
||||
- Chinese markdown files with lines that break mid-sentence at a consistent short width
|
||||
- Files with vertical TOC (single-char-per-line 【】 sections) that need preservation
|
||||
|
||||
## Approach
|
||||
|
||||
Run `scripts/normalize_breaks.py <directory>` — it processes all .md files in the directory.
|
||||
|
||||
The script handles three file patterns:
|
||||
|
||||
1. **Vertical TOC + fixed-width body** — Preserves the decorative single-char TOC section, joins body paragraphs, strips inline page numbers (standalone digits like "3", "4")
|
||||
2. **Outline TOC with stray breaks** — Preserves numbered outline items (一、...、1、...、...... separators), joins body paragraphs
|
||||
3. **Already in paragraph format** — No change (safe to run idempotently)
|
||||
|
||||
### What it preserves
|
||||
|
||||
- Vertical TOC: single CJK/punctuation lines with 【】 brackets
|
||||
- Section headers: 【...】、## ...、# ...、一、二、三、...、1、2、3、...
|
||||
- Outline TOC entries: short numbered lines, lines with ...... separators
|
||||
- Blank lines as paragraph separators
|
||||
|
||||
### What it removes
|
||||
|
||||
- Mid-sentence hard line breaks (joins consecutive CJK body lines)
|
||||
- Inline page numbers (standalone 1-2 digit lines)
|
||||
- Trailing blank lines
|
||||
|
||||
## Beyond the script: bold fragments, conjoined paragraphs, encoding
|
||||
|
||||
The script handles simple fixed-width body text. Some PDF→markdown conversions produce more complex artifacts that need manual multi-pass Python scripts via `execute_code`:
|
||||
|
||||
### Bold marker fragmentation
|
||||
|
||||
`**...text...**` blocks split across blank lines with stray `**` at fragment boundaries:
|
||||
|
||||
```
|
||||
**第三条 特色——依据五大要素，构建次第修学。营造良好氛围，提供有效**
|
||||
|
||||
引导。
|
||||
```
|
||||
|
||||
**Fix**: Join fragments, remove stray `**` from join point, add closing `**` to final result. See `references/bold-fragments.md` for full pattern catalog and multi-pass workflow.
|
||||
|
||||
**Critical pitfall**: Do NOT join lines where BOTH the first and second line are complete bold blocks (start+end with `**`). These are separate entries, not fragments:
|
||||
```
|
||||
**第一条 ...之道。** ← complete bold item
|
||||
← blank line
|
||||
**第二条 ...合一。** ← complete bold item (DON'T JOIN)
|
||||
```
|
||||
|
||||
### Conjoined paragraphs
|
||||
|
||||
Separate paragraphs/sections merged into one line — opposite problem to the script. Common in song lyrics, dense instructional sections. Requires semantic splitting. See `references/bold-fragments.md`.
|
||||
|
||||
### Encoding artifacts
|
||||
|
||||
`川` (U+5DDD) replacing `"` (curly quote) — search-and-replace: `" 道理川` → `"道理"`, `" 自己的川` → `"自己的"`.
|
||||
|
||||
### Multi-pass approach
|
||||
|
||||
1. **Pass 1**: Join word fragments split by blank lines (conservative — only when current line doesn't end with `。！？` or is NOT a complete bold block)
|
||||
2. **Pass 2**: Split obviously conjoined paragraphs (manual string replacements for known patterns)
|
||||
3. **Pass 3**: Fix stray bold markers, encoding artifacts, stray page numbers
|
||||
4. Verify after each pass; revert with `git checkout` if over-aggressive
|
||||
|
||||
### Heuristic pitfalls
|
||||
|
||||
- **Short-line join** (< 15 chars): Over-joins section headers with body, Q&A pairs (`正念是什么？\n\n就是...`). Only use for clear word-fragment continuations.
|
||||
- **Bold-end join**: Lines ending with `**` are ambiguous — either broken bold fragment or complete bold item. Check if the content before `**` forms a complete sentence (ends with `。`).
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **TOC detection boundaries**: The vertical TOC end is detected by finding the first line with 3+ CJK characters. If a page number like "2" sits between TOC and body, it lands in the TOC section — harmless but visible.
|
||||
- **Section headers without markers**: Plain-text section titles (e.g., "生命可以被设计的依据") without 【】 or number prefixes won't be detected as headers. They'll form standalone paragraphs separated by blank lines, which is fine as long as blank lines exist around them.
|
||||
- **Wiki-link TOC files**: Files like a course index with [[wiki links]] are NOT prose and should be excluded. The script has no special handling — skip those files manually or restore from git.
|
||||
- **Not for mixed CJK/English prose**: The script treats any line with CJK characters as body text. Mixed-language documents may need manual review.
|
||||
@@ -1,110 +0,0 @@
|
||||
# Bold fragments & conjoined paragraphs — fix patterns
|
||||
|
||||
From session fixing `静心学堂学员手册.md` (1575→1478 lines, ~100 fixes).
|
||||
|
||||
## Pattern A: Bold marker fragmentation
|
||||
|
||||
**Problem**: `**...text...**` block split across blank line with stray `**` markers:
|
||||
|
||||
```
|
||||
**第三条 特色——依据五大要素，构建次第修学。营造良好氛围，提供有效**
|
||||
|
||||
引导。
|
||||
```
|
||||
|
||||
**Detection**: Line ends with `**`, next non-blank line continues the sentence (does NOT start with `**`).
|
||||
|
||||
**Fix** (Python):
|
||||
```python
|
||||
# curr ends with **, nxt is continuation (no leading **)
|
||||
curr_fixed = curr.rstrip()[:-2].rstrip() # strip trailing **
|
||||
nxt_fixed = nxt.lstrip()
|
||||
if nxt_fixed.endswith('**'):
|
||||
nxt_fixed = nxt_fixed[:-2].rstrip()
|
||||
joined = curr_fixed + nxt_fixed + '**'
|
||||
else:
|
||||
joined = curr_fixed + nxt_fixed # lost closing ** — may need manual fix
|
||||
```
|
||||
|
||||
### Anti-pattern: Complete bold items
|
||||
|
||||
Do NOT join when BOTH lines are complete bold blocks (start+end with `**`):
|
||||
|
||||
```
|
||||
**第一条 ...之道。** ← DON'T JOIN
|
||||
← blank line
|
||||
**第二条 ...合一。** ← DON'T JOIN
|
||||
```
|
||||
|
||||
**Detection**: Both `curr` and `nxt` start with `**` and end with `**`.
|
||||
|
||||
## Pattern B: Conjoined paragraphs (Type 2)
|
||||
|
||||
Separate sections merged into one line. Common cases:
|
||||
|
||||
### Section headers merged with body
|
||||
```
|
||||
导言：这本指引怎么用这本指引是什么这是一本修学地图...
|
||||
```
|
||||
→ Split into:
|
||||
```
|
||||
导言：这本指引怎么用
|
||||
|
||||
这本指引是什么
|
||||
|
||||
这是一本修学地图...
|
||||
```
|
||||
|
||||
### Song titles merged mid-lyrics
|
||||
```
|
||||
...生生世世不再久违《菩提花开》如果你渴求一滴水...
|
||||
```
|
||||
→ Split into:
|
||||
```
|
||||
...生生世世不再久违
|
||||
|
||||
### 《菩提花开》
|
||||
|
||||
如果你渴求一滴水...
|
||||
```
|
||||
|
||||
### List items merged into one line
|
||||
```
|
||||
不在班级群发布...不从事违法活动不在班级平台拉拢...
|
||||
```
|
||||
→ Split into bullet list:
|
||||
```
|
||||
- 不在班级群发布...
|
||||
- 不从事违法活动
|
||||
- 不在班级平台拉拢...
|
||||
```
|
||||
|
||||
**Approach**: Manual string replacements for known patterns. Regex is unreliable for semantic splits.
|
||||
|
||||
## Pattern C: Stray page numbers
|
||||
|
||||
Standalone digits at line ends, often from PDF page number artifacts:
|
||||
- `42`, `43`, `46`, `47` at end of content lines
|
||||
|
||||
**Fix**: Strip trailing digits that aren't part of dates, durations, or course numbers.
|
||||
|
||||
## Pattern D: Encoding artifacts
|
||||
|
||||
`川` (U+5DDD) replacing curly quotes `"` (U+201C/U+201D):
|
||||
```
|
||||
把" 道理川变成" 自己的川 → 把"道理"变成"自己的"
|
||||
```
|
||||
|
||||
**Fix**: Replace `" 道理川` → `"道理"`, `" 自己的川` → `"自己的"`.
|
||||
|
||||
## Multi-pass workflow
|
||||
|
||||
1. **Pass 1 — Join word fragments**: Scan for lines split by blank line where first line doesn't end with `。！？` and neither line is structural (header/list/table). Skip complete bold items.
|
||||
2. **Pass 2 — Split conjoined**: Apply known string replacements for merged sections, song transitions, list items.
|
||||
3. **Pass 3 — Clean artifacts**: Fix stray `**` markers, encoding issues, stray page numbers.
|
||||
4. **Verify**: `git diff` after each pass; `git checkout` if over-aggressive.
|
||||
|
||||
## Rejected heuristics
|
||||
|
||||
- **Short-line join** (< 15 chars): Over-joins section headers (`中级和高级（以后的事）`) with body, and Q&A pairs (`正念是什么？\n\n就是...`). Only use for clear mid-word fragments.
|
||||
- **Blind `**` stripping**: Removes valid bold formatting from complete bold items.
|
||||
@@ -1,168 +0,0 @@
|
||||
"""
|
||||
Fix extraneous line breaks in Chinese markdown files.
|
||||
|
||||
Three file patterns:
|
||||
1. Fixed-width body text (20-25 chars/line) + vertical TOC -> join lines, remove page nums
|
||||
2. Mostly-paragraph with stray breaks + outline TOC -> join broken lines, preserve list items
|
||||
3. Already fine -> skip (idempotent)
|
||||
|
||||
Usage: python3 normalize_breaks.py <directory>
|
||||
"""
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
CJK = re.compile(r'[\u4e00-\u9fff\u3400-\u4dbf\uf900-\ufaff]')
|
||||
CN_PUNCT = '，。！？；：、""''（）《》【】…—～·'
|
||||
NUM_MARKER = re.compile(r'^[一二三四五六七八九十]+[、，,]')
|
||||
DIGIT_MARKER = re.compile(r'^\d+[、.,]')
|
||||
TOC_SEP = re.compile(r'\.{3,}') # "......" separators in outline TOCs
|
||||
|
||||
|
||||
def has_cjk(s):
|
||||
return bool(CJK.search(s))
|
||||
|
||||
|
||||
def is_page_num(line):
|
||||
s = line.strip()
|
||||
return s and s.isdigit() and len(s) <= 2
|
||||
|
||||
|
||||
def is_toc_line(line):
|
||||
"""Vertical TOC: single char, or 【, 】, ·, or solo digit"""
|
||||
s = line.strip()
|
||||
if not s:
|
||||
return False
|
||||
if len(s) == 1 and (has_cjk(s) or s in CN_PUNCT or s in '【】·' or s.isdigit()):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_section_header(line):
|
||||
"""Section headers: 【...】, ## ..., # ..., 一、..., 1、..., or standalone title lines"""
|
||||
s = line.strip()
|
||||
if not s:
|
||||
return False
|
||||
if s.startswith('【') and s.endswith('】'):
|
||||
return True
|
||||
if s.startswith('#'):
|
||||
return True
|
||||
if NUM_MARKER.match(s):
|
||||
return True
|
||||
if DIGIT_MARKER.match(s):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def is_outline_toc_line(line):
|
||||
"""Outline/list TOC: entries separated by ...... or short numbered items"""
|
||||
s = line.strip()
|
||||
if TOC_SEP.search(s):
|
||||
return True
|
||||
m = re.match(r'^(\d+[.、,]|[一二三四五六七八九十]+[、,])\s*\S', s)
|
||||
if m and len(s) < 30:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def find_toc_end(lines):
|
||||
"""Find where the vertical TOC section ends and body text begins."""
|
||||
for i, line in enumerate(lines):
|
||||
s = line.strip()
|
||||
if has_cjk(s) and len([c for c in s if has_cjk(c)]) >= 3:
|
||||
j = i
|
||||
while j > 0 and not lines[j - 1].strip():
|
||||
j -= 1
|
||||
return j
|
||||
return 0
|
||||
|
||||
|
||||
def process_body(lines):
|
||||
"""Join body text lines into paragraphs, preserving section headers and outline items."""
|
||||
result = []
|
||||
buf = []
|
||||
|
||||
def flush():
|
||||
nonlocal buf
|
||||
if buf:
|
||||
joined = ''.join(buf)
|
||||
result.append(joined)
|
||||
buf = []
|
||||
|
||||
for line in lines:
|
||||
s = line.strip()
|
||||
|
||||
if not s:
|
||||
flush()
|
||||
result.append('')
|
||||
continue
|
||||
|
||||
if is_section_header(s):
|
||||
flush()
|
||||
result.append(s)
|
||||
continue
|
||||
|
||||
if is_outline_toc_line(s):
|
||||
flush()
|
||||
result.append(s)
|
||||
continue
|
||||
|
||||
if is_page_num(s):
|
||||
continue
|
||||
|
||||
if has_cjk(s) or (buf and s):
|
||||
buf.append(s)
|
||||
else:
|
||||
flush()
|
||||
result.append(s)
|
||||
|
||||
flush()
|
||||
return result
|
||||
|
||||
|
||||
def process_file(filepath):
|
||||
content = filepath.read_text(encoding='utf-8')
|
||||
lines = content.split('\n')
|
||||
|
||||
toc_end = find_toc_end(lines)
|
||||
|
||||
if toc_end > 10:
|
||||
toc_part = lines[:toc_end]
|
||||
body_part = lines[toc_end:]
|
||||
body_processed = process_body(body_part)
|
||||
new_lines = toc_part + body_processed
|
||||
else:
|
||||
new_lines = process_body(lines)
|
||||
|
||||
cleaned = []
|
||||
prev_blank = False
|
||||
for line in new_lines:
|
||||
is_blank = line.strip() == ''
|
||||
if is_blank and prev_blank:
|
||||
continue
|
||||
cleaned.append(line)
|
||||
prev_blank = is_blank
|
||||
|
||||
while cleaned and cleaned[-1] == '':
|
||||
cleaned.pop()
|
||||
|
||||
new_content = '\n'.join(cleaned) + '\n'
|
||||
|
||||
if new_content != content:
|
||||
filepath.write_text(new_content, encoding='utf-8')
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
workdir = Path(sys.argv[1])
|
||||
files = sorted(workdir.glob('*.md'))
|
||||
|
||||
for f in files:
|
||||
changed = process_file(f)
|
||||
status = 'FIXED' if changed else 'OK'
|
||||
print(f'{status}: {f.name}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -1,322 +0,0 @@
|
||||
---
|
||||
name: pdf-to-docx-conversion
|
||||
description: "Convert flowing text PDFs (Chinese or multi-language) to DOCX with proper fonts, styles, native bullets, lists, and embedded images. Preserves visual hierarchy from PDF font/size/color data."
|
||||
version: 1.0.0
|
||||
author: Claude
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [PDF, DOCX, Documents, python-docx, pymupdf]
|
||||
---
|
||||
|
||||
# PDF-to-DOCX Conversion
|
||||
|
||||
Convert PDF documents (especially flowing text documents in any language, including CJK) into well-structured DOCX files that preserve fonts, sizes, colors, and layout intent.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
```bash
|
||||
pip install pymupdf python-docx
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- **Style-aware**: reads actual font, size, bold, color from PDF spans
|
||||
- **Native bullets**: uses Word `List Bullet` style instead of Wingdings glyphs
|
||||
- **Native numbering**: uses numbered list style for sequential items
|
||||
- **Image extraction**: detects and embeds PDF images into the DOCX
|
||||
- **Verse/poetry handling**: splits merged verse lines at semantic boundaries
|
||||
- **Multi-language**: works with CJK, RTL, and mixed-script documents
|
||||
- **Flowing text**: text flows across pages; no forced page breaks
|
||||
|
||||
## Quick Start
|
||||
|
||||
```python
|
||||
from pdf_to_docx import convert_pdf_to_docx
|
||||
|
||||
convert_pdf_to_docx("input.pdf", "output.docx")
|
||||
```
|
||||
|
||||
## Step-by-Step Workflow
|
||||
|
||||
### 1. Inspect the PDF
|
||||
|
||||
First, dump the PDF to understand its structure:
|
||||
|
||||
```bash
|
||||
python3 << 'PY'
|
||||
import pymupdf
|
||||
doc = pymupdf.open("input.pdf")
|
||||
for pi in range(len(doc)):
|
||||
page = doc[pi]
|
||||
blocks = page.get_text("dict")["blocks"]
|
||||
for block in blocks:
|
||||
if block["type"] != 0: continue
|
||||
for line in block["lines"]:
|
||||
for span in line["spans"]:
|
||||
bbox = span["bbox"]
|
||||
flags = span["flags"]
|
||||
attrs = []
|
||||
if flags & 2**1: attrs.append("I")
|
||||
if flags & 2**4: attrs.append("B")
|
||||
print(f" Y={bbox[1]:.0f} [{span['size']:.1f}pt {'+'.join(attrs) or '-'}] {span['font']} | {span['text']}")
|
||||
PY
|
||||
```
|
||||
|
||||
Key things to identify:
|
||||
- **Fonts used** (map to DOCX fonts)
|
||||
- **Bullet mechanism** (Wingdings? Unicode?)
|
||||
- **Header hierarchy** (what size = section header vs sub-header)
|
||||
- **Numbered lists** (what delimiter: `1)` `1.` `1）`)
|
||||
- **Images** (check `page.get_images()`)
|
||||
- **Special sections** (tables, verses, forms)
|
||||
|
||||
### 2. Configure the Converter
|
||||
|
||||
Create a config dict matching your PDF's patterns:
|
||||
|
||||
```python
|
||||
config = {
|
||||
"fonts": {
|
||||
"title": "STHeitiSC-Medium",
|
||||
"body": "HYShuSongErKW",
|
||||
"page_number": "HelveticaNeue",
|
||||
},
|
||||
"header_sizes": {"section": 18, "sub": 15},
|
||||
"body_size": 12,
|
||||
"bullet_fonts": ["Wingdings", "Wingdings 2", "Wingdings 3"],
|
||||
"page_number_font": "HelveticaNeue",
|
||||
"numbered_patterns": [r'^\d+\)', r'^\d+\.'], # detect numbered items
|
||||
"skip_fonts": ["HelveticaNeue"], # fonts to skip (page numbers)
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Run the Conversion
|
||||
|
||||
```python
|
||||
from pdf_to_docx import PDFToDOCXConverter
|
||||
|
||||
converter = PDFToDOCXConverter(config)
|
||||
converter.convert("input.pdf", "output.docx")
|
||||
```
|
||||
|
||||
## Core Classes
|
||||
|
||||
### PDFToDOCXConverter
|
||||
|
||||
```python
|
||||
class PDFToDOCXConverter:
|
||||
def __init__(self, config=None):
|
||||
self.cfg = config or self._default_config()
|
||||
|
||||
def convert(self, pdf_path: str, docx_path: str):
|
||||
"""Main entry point."""
|
||||
# 1. Extract all spans
|
||||
# 2. Merge Wingdings bullets with body text
|
||||
# 3. Group lines into paragraphs by Y-gap and style changes
|
||||
# 4. Post-process: split compact lists, verses, merged steps
|
||||
# 5. Build DOCX with proper styles
|
||||
# 6. Embed images
|
||||
pass
|
||||
|
||||
def extract_spans(self, pdf_path: str) -> list[dict]:
|
||||
"""Extract all text spans with full style info."""
|
||||
doc = pymupdf.open(pdf_path)
|
||||
raw = []
|
||||
for pi in range(len(doc)):
|
||||
for block in doc[pi].get_text("dict")["blocks"]:
|
||||
if block["type"] != 0: continue
|
||||
for line in block["lines"]:
|
||||
for span in line["spans"]:
|
||||
raw.append({
|
||||
"text": span["text"], "font": span["font"],
|
||||
"size": span["size"], "flags": span["flags"],
|
||||
"color": span["color"], "bbox": span["bbox"],
|
||||
})
|
||||
return raw
|
||||
|
||||
def detect_bullets(self, line: list) -> bool:
|
||||
"""Check if first span in line is a Wingdings bullet."""
|
||||
return "Wingdings" in line[0]["font"]
|
||||
|
||||
def group_paragraphs(self, lines: list) -> list[dict]:
|
||||
"""Group raw lines into logical paragraphs."""
|
||||
# Group by Y-gap threshold (typically 20-30pt)
|
||||
# Break on style change (font change, size > threshold, bold toggle)
|
||||
# Break on attribution lines (——节选自...)
|
||||
# Break on special fonts (STHeitiSC-Light etc.)
|
||||
pass
|
||||
|
||||
def post_process(self, paras: list) -> list[dict]:
|
||||
"""Split merged compact lists and verses."""
|
||||
# See examples below for common patterns
|
||||
pass
|
||||
```
|
||||
|
||||
## Common Post-Processing Patterns
|
||||
|
||||
### Compact Numbered Lists
|
||||
|
||||
When the PDF flows list items together in one paragraph:
|
||||
|
||||
```python
|
||||
def split_compact_list(text: str) -> list[str]:
|
||||
"""Split '1) foo 2) bar 3) baz' into separate items."""
|
||||
parts = re.split(r'(?=\d+\))', text)
|
||||
return [p for p in parts if p.strip()]
|
||||
```
|
||||
|
||||
### Verse / Poetry Lines
|
||||
|
||||
When the PDF merges verse lines that should be on separate lines:
|
||||
|
||||
```python
|
||||
def split_verse(text: str, split_markers: list[str]) -> list[str]:
|
||||
"""Split verse at semantic phrase boundaries.
|
||||
Example markers: ['感恩', '愿', '更愿']"""
|
||||
pattern = '|'.join(f'(?={m})' for m in split_markers)
|
||||
return [p for p in re.split(pattern, text) if p.strip()]
|
||||
```
|
||||
|
||||
### 小组交流流程 / Process Steps
|
||||
|
||||
Split Chinese process steps numbered 一、二、三、etc.:
|
||||
|
||||
```python
|
||||
def split_chinese_steps(text: str) -> list[str]:
|
||||
"""Split '一、foo 二、bar' into separate items."""
|
||||
parts = re.split(r'(?=[一二三四五六七八九十]、)', text)
|
||||
return [p.strip() for p in parts if p.strip()]
|
||||
```
|
||||
|
||||
## DOCX Construction
|
||||
|
||||
### Font Setup (East Asian fonts)
|
||||
|
||||
```python
|
||||
from docx.oxml.ns import qn
|
||||
from docx.oxml import OxmlElement
|
||||
|
||||
def set_east_asian_font(run, name: str):
|
||||
"""Set CJK font properly in python-docx."""
|
||||
rPr = run._element.get_or_add_rPr()
|
||||
rFonts = rPr.find(qn('w:rFonts'))
|
||||
if rFonts is None:
|
||||
rFonts = OxmlElement('w:rFonts')
|
||||
rPr.insert(0, rFonts)
|
||||
rFonts.set(qn('w:eastAsia'), name)
|
||||
rFonts.set(qn('w:ascii'), name)
|
||||
rFonts.set(qn('w:hAnsi'), name)
|
||||
```
|
||||
|
||||
### Bullet Items
|
||||
|
||||
Use native Word bullets, NOT Wingdings characters:
|
||||
|
||||
```python
|
||||
p = doc.add_paragraph(style='List Bullet')
|
||||
p.clear()
|
||||
run = p.add_run("Your bullet text here")
|
||||
set_east_asian_font(run, font_name)
|
||||
```
|
||||
|
||||
### Image Embedding
|
||||
|
||||
```python
|
||||
from docx.shared import Inches
|
||||
|
||||
p = doc.add_paragraph()
|
||||
p.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
||||
run = p.add_run()
|
||||
run.add_picture(image_path, width=Inches(3.5))
|
||||
```
|
||||
|
||||
Extract images from PDF first:
|
||||
|
||||
```python
|
||||
doc = pymupdf.open("input.pdf")
|
||||
for pi in range(len(doc)):
|
||||
images = doc[pi].get_images()
|
||||
for idx, img in enumerate(images):
|
||||
xref = img[0]
|
||||
base = doc.extract_image(xref)
|
||||
with open(f"extracted_{pi}_{idx}.{base['ext']}", 'wb') as f:
|
||||
f.write(base['image'])
|
||||
```
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After conversion, verify the DOCX:
|
||||
|
||||
```bash
|
||||
python3 -c "
|
||||
from docx import Document
|
||||
doc = Document('output.docx')
|
||||
print(f'Paragraphs: {len(doc.paragraphs)}')
|
||||
for i, p in enumerate(doc.paragraphs):
|
||||
style = p.style.name if p.style else '-'
|
||||
txt = p.text[:80]
|
||||
print(f'[{i:2d}] [{style:15s}] {txt}')
|
||||
"
|
||||
```
|
||||
|
||||
Check for:
|
||||
1. [ ] All sections present (count paragraphs)
|
||||
2. [ ] No merged verses or lists
|
||||
3. [ ] Headers are bold + larger size
|
||||
4. [ ] Bullets use `List Bullet` style
|
||||
5. [ ] Numbered items are separate paragraphs
|
||||
6. [ ] Images present in `word/media/`
|
||||
7. [ ] Attribution lines right-aligned/indented
|
||||
8. [ ] No page numbers leaked into body
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Text over-merged | Y-gap threshold too high | Lower gap threshold (e.g. 20 → 15) |
|
||||
| Missing sections | Skipped by font filter | Add font to skip_fonts or remove filter |
|
||||
| Over-split lines | Y-gap threshold too low | Raise gap threshold (e.g. 20 → 30) |
|
||||
| Wingdings boxes | Unicode bullet inserted | Use `style='List Bullet'` instead |
|
||||
| CJK font wrong | East Asian font not set | Use `set_east_asian_font()` helper |
|
||||
| Image missing | Not extracted before DOCX build | Run `extract_images()` first |
|
||||
| Verse mangled | Regex too aggressive | Tune verse splitting pattern |
|
||||
|
||||
## Full Example Script
|
||||
|
||||
See `scripts/convert_pdf_to_docx.py` for a production-ready converter with all patterns pre-configured.
|
||||
|
||||
```python
|
||||
# scripts/convert_pdf_to_docx.py
|
||||
import pymupdf, re
|
||||
from docx import Document
|
||||
from docx.shared import Pt, Cm, Inches, RGBColor
|
||||
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
||||
from docx.oxml.ns import qn
|
||||
from docx.oxml import OxmlElement
|
||||
|
||||
def convert_pdf_to_docx(pdf_path: str, docx_path: str, config: dict = None):
|
||||
"""Convert a flowing text PDF to DOCX."""
|
||||
cfg = config or {}
|
||||
|
||||
# Extract
|
||||
doc_pdf = pymupdf.open(pdf_path)
|
||||
raw = []
|
||||
for pi in range(len(doc_pdf)):
|
||||
for block in doc_pdf[pi].get_text("dict")["blocks"]:
|
||||
if block["type"] != 0: continue
|
||||
for line in block["lines"]:
|
||||
ss = line["spans"]
|
||||
is_b = any("Wingdings" in s["font"] for s in ss[:1])
|
||||
text = "".join(s["text"] for s in ss)
|
||||
dom = ss[1] if (is_b and len(ss) > 1) else ss[0]
|
||||
raw.append(dict(text=text, font=dom["font"], size=dom["size"],
|
||||
bold=bool(dom["flags"] & 2**4), color=dom["color"],
|
||||
x=dom["bbox"][0], y=dom["bbox"][1], is_bullet=is_b))
|
||||
|
||||
# ... (merge bullets, group paragraphs, post-process, build DOCX)
|
||||
|
||||
# Run:
|
||||
# python scripts/convert_pdf_to_docx.py input.pdf output.docx
|
||||
```
|
||||
@@ -1,549 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Production-ready PDF-to-DOCX converter.
|
||||
Handles multi-language text, mixed fonts, bullets, numbered lists, verses,
|
||||
attributions, images, and flowing text across pages.
|
||||
|
||||
Usage:
|
||||
python convert_pdf_to_docx.py input.pdf output.docx
|
||||
|
||||
Requires: pip install pymupdf python-docx
|
||||
"""
|
||||
|
||||
import sys
|
||||
import re
|
||||
import pymupdf
|
||||
from docx import Document
|
||||
from docx.shared import Pt, Cm, Inches, RGBColor
|
||||
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
||||
from docx.oxml.ns import qn
|
||||
from docx.oxml import OxmlElement
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════════════
|
||||
# Config —— tweak these for your PDF's style conventions
|
||||
# ═════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
DEFAULT_CONFIG = {
|
||||
# Font names used for different roles
|
||||
"fonts": {
|
||||
"title": "STHeitiSC-Medium",
|
||||
"body": "HYShuSongErKW",
|
||||
"page_num": "HelveticaNeue",
|
||||
},
|
||||
# Size thresholds (pt) for paragraph classification
|
||||
"thresholds": {
|
||||
"section_header": 18, # —— bold, centered: 【法义】 【思考】 【练习】
|
||||
"sub_header": 14, # —— bold: 一、认识感恩, 【使用说明】
|
||||
"body": 12,
|
||||
},
|
||||
# Fonts to treat as bullets (skipped as glyphs, trigger List Bullet style)
|
||||
"bullet_fonts": ["Wingdings", "Wingdings 2", "Wingdings 3", "Symbol"],
|
||||
# Fonts to skip entirely (page numbers, decorative markers)
|
||||
"skip_fonts": ["HelveticaNeue"],
|
||||
"skip_size_max": 9.5,
|
||||
# Numbered-list delimiters in the PDF text
|
||||
"numbered_patterns": [
|
||||
r'^\d+\)', # 1) 2) 3)
|
||||
r'^\d+\.\s*', # 1. 2. 3.
|
||||
r'^\d+）', # 1） 2） 3） (full-width parens)
|
||||
r'^\d+、', # 1、 2、 3、 (ideographic comma)
|
||||
],
|
||||
# Paragraph grouping: maximum Y-gap (pt) between lines to keep in same paragraph
|
||||
"y_gap_threshold": 20,
|
||||
# Indentation for body/numbered items (cm)
|
||||
"indent_body": 0.8,
|
||||
# Verse markers —— used to split merged poetic lines
|
||||
"verse_markers": [
|
||||
r'感恩(?!恩)', # 感恩 (not followed by 恩)
|
||||
r'愿我们', # 愿我们
|
||||
r'更愿', # 更愿
|
||||
r'愿人们', # 愿人们
|
||||
r'愿世界', # 愿世界
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════════════
|
||||
# DOCX helpers
|
||||
# ═══════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def set_east_asian_font(run, name: str):
|
||||
"""Set CJK/RTL font properly in python-docx (East Asian + ascii + hAnsi)."""
|
||||
run.font.name = name
|
||||
rPr = run._element.get_or_add_rPr()
|
||||
rFonts = rPr.find(qn('w:rFonts'))
|
||||
if rFonts is None:
|
||||
rFonts = OxmlElement('w:rFonts')
|
||||
rPr.insert(0, rFonts)
|
||||
for attr in ('eastAsia', 'ascii', 'hAnsi'):
|
||||
rFonts.set(qn(f'w:{attr}'), name)
|
||||
|
||||
|
||||
def _mk_color(val: int) -> RGBColor | None:
|
||||
if val and val != 0:
|
||||
return RGBColor((val >> 16) & 0xFF, (val >> 8) & 0xFF, val & 0xFF)
|
||||
return None
|
||||
|
||||
|
||||
def _is_numbered(text: str, cfg: dict) -> bool:
|
||||
return any(re.match(pat, text) for pat in cfg["numbered_patterns"])
|
||||
|
||||
|
||||
def _is_bullet_font(font: str, cfg: dict) -> bool:
|
||||
return any(b in font for b in cfg["bullet_fonts"])
|
||||
|
||||
|
||||
def _skip_span(span: dict, cfg: dict) -> bool:
|
||||
"""True if this span should be dropped entirely."""
|
||||
if span["font"] in cfg["skip_fonts"] and span["size"] <= cfg["skip_size_max"]:
|
||||
return True
|
||||
if _is_bullet_font(span["font"], cfg):
|
||||
return False # bullets are handled upstream
|
||||
return False
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════════════
|
||||
# Extraction
|
||||
# ════════════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def extract_lines(pdf_path: str, cfg: dict) -> list[dict]:
|
||||
"""Return flattened list of text lines with style info."""
|
||||
doc_pdf = pymupdf.open(pdf_path)
|
||||
lines = []
|
||||
for pi in range(len(doc_pdf)):
|
||||
page = doc_pdf[pi]
|
||||
for block in page.get_text("dict")["blocks"]:
|
||||
if block["type"] != 0:
|
||||
continue # skip images
|
||||
for line in block["lines"]:
|
||||
spans = line["spans"]
|
||||
if not spans:
|
||||
continue
|
||||
|
||||
# Detect bullet: first span is Wingdings / Symbol
|
||||
is_bullet = _is_bullet_font(spans[0]["font"], cfg)
|
||||
|
||||
# Dominant span for style (skip Wingdings glyph)
|
||||
dom = spans[1] if (is_bullet and len(spans) > 1) else spans[0]
|
||||
|
||||
# Skip decorative spans entirely
|
||||
if _skip_span(dom, cfg):
|
||||
continue
|
||||
|
||||
text = "".join(s["text"] for s in spans)
|
||||
lines.append({
|
||||
"text": text,
|
||||
"font": dom["font"],
|
||||
"size": dom["size"],
|
||||
"bold": bool(dom["flags"] & 2**4),
|
||||
"color": dom["color"],
|
||||
"x": dom["bbox"][0],
|
||||
"y": dom["bbox"][1],
|
||||
"is_bullet": is_bullet,
|
||||
})
|
||||
return lines
|
||||
|
||||
|
||||
def extract_images(pdf_path: str, out_dir: str = "/tmp") -> list[str]:
|
||||
"""Extract all embedded images from PDF. Returns list of file paths."""
|
||||
doc = pymupdf.open(pdf_path)
|
||||
paths = []
|
||||
for pi in range(len(doc)):
|
||||
page = doc[pi]
|
||||
for idx, img in enumerate(page.get_images()):
|
||||
xref = img[0]
|
||||
base = doc.extract_image(xref)
|
||||
path = f"{out_dir}/pdf_img_p{pi}_{idx}.{base['ext']}"
|
||||
with open(path, "wb") as f:
|
||||
f.write(base["image"])
|
||||
paths.append((pi, path))
|
||||
return paths
|
||||
|
||||
|
||||
# ═════════════════════════════════════════════════════════════════════════════════════════════
|
||||
# Grouping & Classification
|
||||
# ══════════════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def group_paragraphs(lines: list[dict], cfg: dict) -> list[dict]:
|
||||
"""Group raw lines into logical paragraphs."""
|
||||
paras = []
|
||||
i = 0
|
||||
gap_thresh = cfg["y_gap_threshold"]
|
||||
|
||||
while i < len(lines):
|
||||
ln = lines[i]
|
||||
|
||||
# ─── Headers ───
|
||||
if ln["bold"] and ln["size"] >= cfg["thresholds"]["section_header"]:
|
||||
paras.append({
|
||||
"text": ln["text"], "font": ln["font"], "size": ln["size"],
|
||||
"bold": True, "color": ln["color"], "x": ln["x"],
|
||||
"kind": "header"
|
||||
})
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if ln["bold"] and ln["size"] >= cfg["thresholds"]["sub_header"]:
|
||||
paras.append({
|
||||
"text": ln["text"], "font": ln["font"], "size": ln["size"],
|
||||
"bold": True, "color": ln["color"], "x": ln["x"],
|
||||
"kind": "subheader"
|
||||
})
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# ─── Attribution ───
|
||||
if ln["text"].startswith("——"):
|
||||
paras.append({
|
||||
"text": ln["text"], "font": ln["font"], "size": ln["size"],
|
||||
"bold": False, "color": ln["color"], "x": ln["x"],
|
||||
"kind": "attribution"
|
||||
})
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# ─── Bullet item ───
|
||||
if ln["is_bullet"]:
|
||||
body = ln["text"].lstrip("\uf06c \uf0b7 \u2022 ").lstrip() # strip common bullet chars
|
||||
buf = [body]
|
||||
bf, bs = ln["font"], ln["size"]
|
||||
i += 1
|
||||
while i < len(lines):
|
||||
nxt = lines[i]
|
||||
if nxt["bold"] and nxt["size"] >= cfg["thresholds"]["sub_header"]:
|
||||
break
|
||||
if nxt["is_bullet"]:
|
||||
break
|
||||
if nxt["text"].startswith("——"):
|
||||
break
|
||||
gap = nxt["y"] - (lines[i - 1]["y"] + lines[i - 1]["size"])
|
||||
if gap > gap_thresh:
|
||||
break
|
||||
if _is_numbered(nxt["text"], cfg) and nxt["x"] <= 115:
|
||||
break # nested numbered item = new para
|
||||
if nxt["font"] != bf:
|
||||
break
|
||||
buf.append(nxt["text"])
|
||||
i += 1
|
||||
paras.append({
|
||||
"text": "".join(buf), "font": bf, "size": bs,
|
||||
"bold": False, "color": ln["color"], "x": ln["x"],
|
||||
"kind": "bullet"
|
||||
})
|
||||
continue
|
||||
|
||||
# ─── Special fonts (one-liners like STHeitiSC-Light notes) ───
|
||||
if ln["font"] == "STHeitiSC-Light":
|
||||
paras.append({
|
||||
"text": ln["text"], "font": cfg["fonts"]["body"], "size": ln["size"],
|
||||
"bold": False, "color": ln["color"], "x": ln["x"],
|
||||
"kind": "special"
|
||||
})
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# ─── Body / numbered / exercise labels ───
|
||||
buf = [ln["text"]]
|
||||
bf, bs, bc, bx = ln["font"], ln["size"], ln["color"], ln["x"]
|
||||
i += 1
|
||||
while i < len(lines):
|
||||
nxt = lines[i]
|
||||
# Hard breaks
|
||||
if nxt["bold"] and nxt["size"] >= cfg["thresholds"]["sub_header"]:
|
||||
break
|
||||
if nxt["is_bullet"]:
|
||||
break
|
||||
if nxt["text"].startswith("——"):
|
||||
break
|
||||
if nxt["font"] == "STHeitiSC-Light":
|
||||
break
|
||||
if _skip_span(nxt, cfg):
|
||||
i += 1
|
||||
continue
|
||||
|
||||
gap = nxt["y"] - (lines[i - 1]["y"] + lines[i - 1]["size"])
|
||||
style_changed = nxt["font"] != bf or abs(nxt["size"] - bs) > 1.5
|
||||
|
||||
# Break on new numbered item at left margin
|
||||
is_new_numbered = _is_numbered(nxt["text"], cfg) and nxt["x"] <= 115
|
||||
# Break on exercise day headers
|
||||
is_day = bool(re.match(r'^第\d+ 天', nxt["text"]))
|
||||
is_ex_label = bool(re.match(r'^(今日感恩练习心得|感恩日记|我的练习)', nxt["text"]))
|
||||
|
||||
if gap > gap_thresh or style_changed or is_new_numbered or is_day or is_ex_label:
|
||||
break
|
||||
buf.append(nxt["text"])
|
||||
i += 1
|
||||
|
||||
text = "".join(buf)
|
||||
kind = "body"
|
||||
if _is_numbered(text, cfg) and bx <= 115:
|
||||
kind = "numbered"
|
||||
elif bool(re.match(r'^第\d+ 天', text)):
|
||||
kind = "day_header"
|
||||
elif bool(re.match(r'^(今日感恩练习心得|感恩日记|我的练习)', text)):
|
||||
kind = "exercise_label"
|
||||
elif bx > 160:
|
||||
kind = "centered_body"
|
||||
|
||||
paras.append({
|
||||
"text": text, "font": bf, "size": bs, "bold": False,
|
||||
"color": bc, "x": bx, "kind": kind
|
||||
})
|
||||
|
||||
return paras
|
||||
|
||||
|
||||
# ══════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
# Post-processing
|
||||
# ════════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def split_compact_lists(paras: list[dict], cfg: dict) -> list[dict]:
|
||||
"""Split paragraphs that contain multiple numbered items."""
|
||||
out = []
|
||||
for p in paras:
|
||||
text = p["text"]
|
||||
numbers = re.findall(r'\d+\)', text)
|
||||
# Only split if more than 2 numbered items in a body paragraph
|
||||
if p["kind"] in ("numbered", "body") and len(numbers) > 2:
|
||||
parts = re.split(r'(?=\d+\))', text)
|
||||
for part in parts:
|
||||
if part.strip():
|
||||
out.append({
|
||||
"text": part.strip(), "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"], "kind": "numbered"
|
||||
})
|
||||
else:
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
def split_verses(paras: list[dict], cfg: dict) -> list[dict]:
|
||||
"""Split merged poetic / verse lines."""
|
||||
out = []
|
||||
for p in paras:
|
||||
text = p["text"]
|
||||
markers = cfg.get("verse_markers", [])
|
||||
if not markers:
|
||||
out.append(p)
|
||||
continue
|
||||
|
||||
# Heuristic: paragraph contains repeated marker phrases
|
||||
total_markers = sum(len(re.findall(m, text)) for m in markers)
|
||||
if total_markers < 3:
|
||||
out.append(p)
|
||||
continue
|
||||
|
||||
# Build a combined split regex from all markers
|
||||
combined = '|'.join(f'(?={m})' for m in markers)
|
||||
parts = re.split(combined, text)
|
||||
# Also split Chinese process steps (一、二、三、) that may prefix the verse
|
||||
prefix = ""
|
||||
verse_start = 0
|
||||
for idx, part in enumerate(parts):
|
||||
if re.match(r'[一二三四五六七八九十]、', part):
|
||||
prefix += part
|
||||
verse_start = idx + 1
|
||||
else:
|
||||
break
|
||||
|
||||
# Emit prefix steps
|
||||
if prefix:
|
||||
for step in re.split(r'(?=[一二三四五六七八九十]、)', prefix):
|
||||
if step.strip():
|
||||
out.append({
|
||||
"text": step.strip(), "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"], "kind": "numbered"
|
||||
})
|
||||
|
||||
# Emit verse lines
|
||||
for part in parts[verse_start:]:
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
# Check for trailing process step (五、回向 etc.)
|
||||
tail_match = re.search(r'([一二三四五六七八九十]、.+)$', part)
|
||||
if tail_match:
|
||||
main_text = part[:tail_match.start()].strip()
|
||||
tail = tail_match.group(1)
|
||||
if main_text:
|
||||
out.append({
|
||||
"text": main_text, "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"] + 100, "kind": "verse_line"
|
||||
})
|
||||
out.append({
|
||||
"text": tail, "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"], "kind": "numbered"
|
||||
})
|
||||
else:
|
||||
out.append({
|
||||
"text": part, "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"] + 100, "kind": "verse_line"
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def split_chinese_steps(paras: list[dict]) -> list[dict]:
|
||||
"""Split merged Chinese process steps (一、二、etc.) in body paragraphs."""
|
||||
out = []
|
||||
for p in paras:
|
||||
text = p["text"]
|
||||
if p["kind"] == "body" and len(re.findall(r'[一二三四五六七八九十]、', text)) > 1:
|
||||
parts = re.split(r'(?=[一二三四五六七八九十]、)', text)
|
||||
for part in parts:
|
||||
if part.strip():
|
||||
out.append({
|
||||
"text": part.strip(), "font": p["font"], "size": p["size"],
|
||||
"bold": False, "color": p["color"], "x": p["x"], "kind": "numbered"
|
||||
})
|
||||
else:
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
# ═════════════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
# DOCX building
|
||||
# ═════════════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def build_docx(paras: list[dict], images: list[tuple[int, str]], cfg: dict) -> Document:
|
||||
"""Build a DOCX from classified paragraphs."""
|
||||
doc = Document()
|
||||
section = doc.sections[0]
|
||||
section.page_width = Cm(21.0)
|
||||
section.page_height = Cm(29.7)
|
||||
section.top_margin = Cm(2.54)
|
||||
section.bottom_margin = Cm(2.54)
|
||||
section.left_margin = Cm(3.18)
|
||||
section.right_margin = Cm(3.18)
|
||||
|
||||
# Track which images have been inserted (insert after first occurrence)
|
||||
inserted_images = set()
|
||||
|
||||
def _add_run(paragraph, text: str, font: str, size: float, bold: bool = False,
|
||||
color=None, alignment=None):
|
||||
if alignment is not None:
|
||||
paragraph.alignment = alignment
|
||||
run = paragraph.add_run(text)
|
||||
run.font.size = Pt(size)
|
||||
run.font.bold = bold
|
||||
if color:
|
||||
run.font.color.rgb = color
|
||||
set_east_asian_font(run, font)
|
||||
return run
|
||||
|
||||
def _add_para(text: str, font: str, size: float, bold: bool = False,
|
||||
alignment=None, sb: int = 0, sa: int = 0,
|
||||
color=None, style=None, indent: float = None):
|
||||
if style:
|
||||
p = doc.add_paragraph(style=style)
|
||||
p.clear()
|
||||
else:
|
||||
p = doc.add_paragraph()
|
||||
p.paragraph_format.space_before = Pt(sb)
|
||||
p.paragraph_format.space_after = Pt(sa)
|
||||
p.paragraph_format.line_spacing = 1.15
|
||||
if indent:
|
||||
p.paragraph_format.left_indent = Cm(indent)
|
||||
_add_run(p, text, font, size, bold, color, alignment)
|
||||
return p
|
||||
|
||||
# Image helper
|
||||
def _insert_image(image_path: str):
|
||||
ip = doc.add_paragraph()
|
||||
ip.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
||||
ip.paragraph_format.space_before = Pt(6)
|
||||
ip.paragraph_format.space_after = Pt(6)
|
||||
ir = ip.add_run()
|
||||
ir.add_picture(image_path, width=Inches(3.3))
|
||||
|
||||
for p in paras:
|
||||
text = p["text"]
|
||||
font = p["font"]
|
||||
size = p["size"]
|
||||
bold = p["bold"]
|
||||
color = _mk_color(p["color"])
|
||||
kind = p["kind"]
|
||||
|
||||
if kind == "header":
|
||||
_add_para(text, font, size, bold=True,
|
||||
alignment=WD_ALIGN_PARAGRAPH.CENTER, sb=10, sa=8)
|
||||
elif kind == "subheader":
|
||||
_add_para(text, font, size, bold=True, sb=8, sa=4)
|
||||
elif kind == "bullet":
|
||||
_add_para(text, font, size, style='List Bullet', sb=0, sa=1, color=color)
|
||||
elif kind == "attribution":
|
||||
_add_para(text, font, size,
|
||||
alignment=WD_ALIGN_PARAGRAPH.RIGHT, sb=2, sa=6, color=color)
|
||||
elif kind == "numbered":
|
||||
_add_para(text, font, size, indent=cfg["indent_body"], sb=1, sa=1, color=color)
|
||||
elif kind == "day_header":
|
||||
_add_para(text, font, size, bold=True, sb=6, sa=2, color=color)
|
||||
elif kind == "exercise_label":
|
||||
_add_para(text, font, size, sb=2, sa=1, color=color)
|
||||
elif kind == "special":
|
||||
_add_para(text, font, size, sb=6, sa=4)
|
||||
elif kind == "verse_line":
|
||||
_add_para(text, font, size, indent=2.0, sb=0, sa=0, color=color)
|
||||
elif kind == "centered_body":
|
||||
_add_para(text, font, size,
|
||||
alignment=WD_ALIGN_PARAGRAPH.CENTER, sb=2, sa=4, color=color)
|
||||
else: # body
|
||||
indent = cfg["indent_body"] if p["x"] > 105 else None
|
||||
_add_para(text, font, size, indent=indent, sb=1, sa=2, color=color)
|
||||
|
||||
# Insert images after paragraphs containing "参考示例" or other markers
|
||||
if "参考示例" in text or "示例" in text:
|
||||
for pi, img_path in images:
|
||||
if img_path not in inserted_images:
|
||||
_insert_image(img_path)
|
||||
inserted_images.add(img_path)
|
||||
break
|
||||
|
||||
return doc
|
||||
|
||||
|
||||
# ════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
# Public API
|
||||
# ════════════════════════════════════════════════════════════════════════════════════════════════════════════
|
||||
|
||||
def convert_pdf_to_docx(pdf_path: str, docx_path: str, config: dict = None):
|
||||
"""
|
||||
Convert a flowing text PDF to a well-structured DOCX.
|
||||
|
||||
Args:
|
||||
pdf_path: Path to input PDF
|
||||
docx_path: Path to output DOCX
|
||||
config: Optional override dict (merged with DEFAULT_CONFIG)
|
||||
"""
|
||||
cfg = DEFAULT_CONFIG.copy()
|
||||
if config:
|
||||
cfg.update(config)
|
||||
|
||||
# 1. Extract images
|
||||
images = extract_images(pdf_path)
|
||||
|
||||
# 2. Extract text lines
|
||||
lines = extract_lines(pdf_path, cfg)
|
||||
|
||||
# 3. Group into paragraphs
|
||||
paras = group_paragraphs(lines, cfg)
|
||||
|
||||
# 4. Post-process
|
||||
paras = split_compact_lists(paras, cfg)
|
||||
paras = split_verses(paras, cfg)
|
||||
paras = split_chinese_steps(paras)
|
||||
|
||||
# 5. Build DOCX
|
||||
doc = build_docx(paras, images, cfg)
|
||||
doc.save(docx_path)
|
||||
print(f"Saved: {docx_path} ({len(doc.paragraphs)} paragraphs)")
|
||||
return docx_path
|
||||
|
||||
|
||||
# CLI
|
||||
if __name__ == "__main__":
|
||||
if len(sys.argv) != 3:
|
||||
print("Usage: python convert_pdf_to_docx.py <input.pdf> <output.docx>")
|
||||
sys.exit(1)
|
||||
convert_pdf_to_docx(sys.argv[1], sys.argv[2])
|
||||
@@ -1,31 +0,0 @@
|
||||
{
|
||||
"_description": "Pre-configured for Chinese Sutra/study-material PDFs with STHeitiSC-HYShuSongErKW fonts",
|
||||
"fonts": {
|
||||
"title": "STHeitiSC-Medium",
|
||||
"body": "HYShuSongErKW",
|
||||
"page_num": "HelveticaNeue"
|
||||
},
|
||||
"thresholds": {
|
||||
"section_header": 18,
|
||||
"sub_header": 14,
|
||||
"body": 12
|
||||
},
|
||||
"bullet_fonts": ["Wingdings", "Wingdings 2", "Wingdings 3", "Symbol"],
|
||||
"skip_fonts": ["HelveticaNeue", "Arial-BoldMT", "ArialMT"],
|
||||
"skip_size_max": 9.9,
|
||||
"numbered_patterns": [
|
||||
"^\\d+\\)",
|
||||
"^\\d+\\.\\s*",
|
||||
"^\\d+）",
|
||||
"^\\d+、"
|
||||
],
|
||||
"y_gap_threshold": 20,
|
||||
"indent_body": 0.8,
|
||||
"verse_markers": [
|
||||
"感恩(?!恩)",
|
||||
"愿我们",
|
||||
"更愿",
|
||||
"愿人们",
|
||||
"愿世界"
|
||||
]
|
||||
}
|
||||
@@ -1,84 +0,0 @@
|
||||
---
|
||||
name: pptx-translate
|
||||
description: Translate PowerPoint files between Chinese and English — extract strings to YAML, translate, quality review, and write back with font-shrink + auto-fit for layout.
|
||||
category: productivity
|
||||
---
|
||||
|
||||
# PPTX Translation
|
||||
|
||||
Translate `.pptx` files between Chinese and English. Covers the full pipeline: extraction → translation → review → write-back.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Extract strings to YAML
|
||||
|
||||
Run `scripts/extract.py original.pptx strings.yaml`. Produces YAML with entries:
|
||||
|
||||
```yaml
|
||||
- slide: 1
|
||||
shape: 0
|
||||
run: 0
|
||||
kind: title
|
||||
zh: 开启生命的富足
|
||||
en: ""
|
||||
```
|
||||
|
||||
- `slide` — 1-based slide number
|
||||
- `shape` — 0-based shape index within slide
|
||||
- `run` — 0-based paragraph index within text frame (or computed index for tables)
|
||||
- `kind` — `title` | `subtitle` | `center_title` | `body` | `table` | `notes`
|
||||
- `zh` — source text
|
||||
- `en` — translation target (initially empty)
|
||||
|
||||
Table `run` index formula: `num_rows * col + row`. Reverse with `row = run % num_rows`, `col = run // num_rows`.
|
||||
|
||||
Speaker notes use `shape: -1`.
|
||||
|
||||
### 2. Translate
|
||||
|
||||
**Do NOT call external translation APIs.** Translate directly — the agent IS the model. The user corrects this: "Why do you call external models to do it? You can do it yourself!"
|
||||
|
||||
Fill in the `en` field for every entry. Batch if needed, but translate in your response, not via API calls.
|
||||
|
||||
Terminology guidance for Buddhist/gratitude content:
|
||||
- 感恩=gratitude, 缘起=dependent origination, 众生=sentient beings
|
||||
- 因缘=causes and conditions, 三宝=Three Jewels, 福报=merit/blessings
|
||||
- 座上=formal practice, 座下=daily life practice, 共修=group practice
|
||||
- 上报四重恩=repaying the four great kindnesses
|
||||
|
||||
### 3. Quality review
|
||||
|
||||
Scan for:
|
||||
- Terminology consistency (same zh term → same en term throughout)
|
||||
- Ellipsis convention — English uses 3 dots `...`, zh may use 6
|
||||
- Buddhist term accuracy
|
||||
- Missing translations
|
||||
- Overly literal renderings
|
||||
|
||||
### 4. Write back with layout fixes
|
||||
|
||||
Run `scripts/build.py strings.yaml original.pptx translated.pptx`.
|
||||
|
||||
The script:
|
||||
- Replaces text in matching paragraphs (clears all runs, sets first run)
|
||||
- Replaces table cell text (using row/col from computed index)
|
||||
- Reduces font size by 18% (`FONT_SCALE = 0.82`) on all translated shapes and tables
|
||||
- Sets `auto_size = TEXT_TO_FIT_SHAPE` on text frames to handle overflow
|
||||
- English text is ~1.3–1.5× longer than Chinese — font shrink + auto-fit handles most cases
|
||||
|
||||
## Alternate scripts
|
||||
|
||||
The absorbed `pptx-translation` skill had alternate script names: `extract_pptx.py` and `build_pptx.py`. These are functionally equivalent to `extract.py` and `build.py` with minor formatting differences (docstrings, variable naming). If the primary scripts fail, the alternates are available in the archive at `~/.hermes/skills/.archive/pptx-translation/scripts/`.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- "run" in the YAML is actually the **paragraph index** within a text frame, not the OOXML text-run index. python-pptx iterates paragraphs, not runs.
|
||||
- Font shrink only applies to runs that have an explicit `font.size` — inherited sizes from paragraph/layout defaults are skipped.
|
||||
- After write-back, verify with `python -m markitdown translated.pptx` to check text landed correctly.
|
||||
- Tables: font shrink is applied per-cell text frame. Each cell is its own text frame.
|
||||
- markitdown may fail with `ModuleNotFoundError: dotenv` — run `pip install python-dotenv` first.
|
||||
|
||||
## Scripts
|
||||
|
||||
- `scripts/extract.py` — extract strings from PPTX to YAML
|
||||
- `scripts/build.py` — write translations back with font shrink + auto-fit
|
||||
@@ -1,68 +0,0 @@
|
||||
import sys, yaml
|
||||
from pptx import Presentation
|
||||
from pptx.util import Pt
|
||||
from pptx.enum.text import MSO_AUTO_SIZE
|
||||
|
||||
FONT_SCALE = 0.82 # shrink ~18%
|
||||
|
||||
yaml_path, src_path, out_path = sys.argv[1], sys.argv[2], sys.argv[3]
|
||||
|
||||
with open(yaml_path) as f:
|
||||
entries = yaml.safe_load(f)
|
||||
|
||||
index = {}
|
||||
for e in entries:
|
||||
index[(e["slide"], e["shape"], e["run"])] = e["en"]
|
||||
|
||||
|
||||
def shrink_font_tf(tf):
|
||||
for para in tf.paragraphs:
|
||||
for run in para.runs:
|
||||
if run.font.size:
|
||||
run.font.size = Pt(int(run.font.size.pt * FONT_SCALE))
|
||||
try:
|
||||
tf.auto_size = MSO_AUTO_SIZE.TEXT_TO_FIT_SHAPE
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
prs = Presentation(src_path)
|
||||
|
||||
for slide_num, slide in enumerate(prs.slides, 1):
|
||||
for shape_idx, shape in enumerate(slide.shapes):
|
||||
if shape.has_text_frame:
|
||||
has_translation = False
|
||||
for para_idx, para in enumerate(shape.text_frame.paragraphs):
|
||||
key = (slide_num, shape_idx, para_idx)
|
||||
if key in index:
|
||||
has_translation = True
|
||||
en = index[key]
|
||||
for r in para.runs:
|
||||
r.text = ""
|
||||
if para.runs:
|
||||
para.runs[0].text = en
|
||||
else:
|
||||
para.add_run().text = en
|
||||
if has_translation:
|
||||
shrink_font_tf(shape.text_frame)
|
||||
|
||||
elif shape.has_table:
|
||||
num_rows = len(shape.table.rows)
|
||||
for r in range(num_rows * len(shape.table.columns)):
|
||||
key = (slide_num, shape_idx, r)
|
||||
if key in index:
|
||||
row = r % num_rows
|
||||
col = r // num_rows
|
||||
shape.table.cell(row, col).text = index[key]
|
||||
for row in shape.table.rows:
|
||||
for cell in row.cells:
|
||||
shrink_font_tf(cell.text_frame)
|
||||
|
||||
key = (slide_num, -1, 0)
|
||||
if key in index and slide.has_notes_slide:
|
||||
ns = slide.notes_slide
|
||||
ns.notes_text_frame.clear()
|
||||
ns.notes_text_frame.paragraphs[0].add_run().text = index[key]
|
||||
|
||||
prs.save(out_path)
|
||||
print(f"Saved {out_path}")
|
||||
@@ -1,68 +0,0 @@
|
||||
import sys, yaml
|
||||
from pptx import Presentation
|
||||
|
||||
PLACEHOLDER_KINDS = {
|
||||
1: "title", 2: "body", 3: "center_title",
|
||||
4: "subtitle", 5: "body", 6: "body", 7: "body",
|
||||
}
|
||||
|
||||
|
||||
def shape_kind(shape):
|
||||
if shape.has_table:
|
||||
return "table"
|
||||
try:
|
||||
ph = shape.placeholder_format
|
||||
if ph is not None and ph.type is not None:
|
||||
return PLACEHOLDER_KINDS.get(ph.type, "body")
|
||||
except ValueError:
|
||||
pass
|
||||
return "body"
|
||||
|
||||
|
||||
def extract(pptx_path):
|
||||
prs = Presentation(pptx_path)
|
||||
entries = []
|
||||
|
||||
for slide_num, slide in enumerate(prs.slides, 1):
|
||||
for shape_idx, shape in enumerate(slide.shapes):
|
||||
kind = shape_kind(shape)
|
||||
|
||||
if shape.has_text_frame:
|
||||
for para_idx, para in enumerate(shape.text_frame.paragraphs):
|
||||
full = para.text.strip()
|
||||
if not full:
|
||||
continue
|
||||
entries.append({
|
||||
"slide": slide_num, "shape": shape_idx,
|
||||
"run": para_idx, "kind": kind,
|
||||
"zh": full, "en": "",
|
||||
})
|
||||
|
||||
elif shape.has_table:
|
||||
for row_idx, row in enumerate(shape.table.rows):
|
||||
for col_idx, cell in enumerate(row.cells):
|
||||
text = cell.text.strip()
|
||||
if not text:
|
||||
continue
|
||||
entries.append({
|
||||
"slide": slide_num, "shape": shape_idx,
|
||||
"run": len(shape.table.rows) * col_idx + row_idx,
|
||||
"kind": "table", "zh": text, "en": "",
|
||||
})
|
||||
|
||||
if slide.has_notes_slide:
|
||||
notes = slide.notes_slide.notes_text_frame.text.strip()
|
||||
if notes:
|
||||
entries.append({
|
||||
"slide": slide_num, "shape": -1, "run": 0,
|
||||
"kind": "notes", "zh": notes, "en": "",
|
||||
})
|
||||
|
||||
return entries
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
entries = extract(sys.argv[1])
|
||||
with open(sys.argv[2], "w") as f:
|
||||
yaml.dump(entries, f, allow_unicode=True, default_flow_style=False, sort_keys=False)
|
||||
print(f"Extracted {len(entries)} entries to {sys.argv[2]}")
|
||||
@@ -1,26 +0,0 @@
|
||||
# MPI Skills
|
||||
|
||||
These skills live in this directory and are loaded via `~/.hermes/config.yaml`.
|
||||
|
||||
## Install for Hermes
|
||||
|
||||
Add to `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
skills:
|
||||
external_dirs:
|
||||
- $MPI_PROJECT_ROOT/skills
|
||||
```
|
||||
|
||||
{% Edit config.yaml directly — `hermes config set` stores list values as strings. %}
|
||||
|
||||
## Skills
|
||||
|
||||
| Name | What it does |
|
||||
|---|---|
|
||||
| `translation` | Translate Chinese↔English Buddhist/Dharma content |
|
||||
| `terms-search` | Full-text search across MPI term database |
|
||||
| `translation-review` | Review translations for quality issues |
|
||||
| `chinese-text-normalize` | Normalize Chinese markdown line breaks |
|
||||
| `pptx-translate` | Translate PowerPoint files |
|
||||
| `pdf-to-docx-conversion` | Convert PDFs to DOCX with layout preservation |
|
||||
@@ -1,88 +0,0 @@
|
||||
---
|
||||
name: terms-search
|
||||
description: Full-text search across the MPI term database. Use when translating or looking up Chinese-English Buddhist/MPI terminology.
|
||||
category: research
|
||||
---
|
||||
|
||||
# Terms Search
|
||||
|
||||
Database: `$MPI_PROJECT_ROOT/terms-search/termlib.duckdb`
|
||||
CLI: `$MPI_PROJECT_ROOT/terms-search/search.py`
|
||||
Server: `$MPI_PROJECT_ROOT/terms-search/server.py`
|
||||
|
||||
## CLI (preferred)
|
||||
|
||||
```
|
||||
$MPI_PROJECT_ROOT/terms-search/search.py <query> [limit]
|
||||
```
|
||||
|
||||
Multi-word queries are ANDed. Searches both `zh` and `en` columns.
|
||||
|
||||
## Python module
|
||||
|
||||
```python
|
||||
import sys
|
||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
||||
from search import search
|
||||
results = search("空性", limit=5, src="DoT定稿")
|
||||
# → list of {zh, en, loc, source} dicts
|
||||
```
|
||||
|
||||
Use this inside `execute_code` scripts for batch lookups — no subprocess needed.
|
||||
|
||||
## HTTP API (use only when CLI is insufficient)
|
||||
|
||||
Start: `python3 $MPI_PROJECT_ROOT/terms-search/server.py` (port 8910)
|
||||
|
||||
- `GET /` — plain HTML UI (form + results table, no CSS)
|
||||
- `GET /` — plain HTML UI (form + results table, no CSS)
|
||||
- `GET /search?q=...&loc=...&src=...&limit=...` — JSON `{count, results: [{zh, en, loc, source}]}`
|
||||
- `GET /sources` — JSON array of `{source, count}` for all source tables
|
||||
|
||||
All params optional. Omit `limit` for all results. Query terms are ANDed across zh+en.
|
||||
Errors return `{"error": "..."}` with HTTP 500 (API) or shown inline (UI).
|
||||
|
||||
## Source tables
|
||||
|
||||
| src | rows | description |
|
||||
|---|---|---|
|
||||
| BAICKZ | 7,679 | Main term bank with example sentences |
|
||||
| 佛教术语 | 1,795 | Buddhist terminology from 定稿书目术语库 |
|
||||
| DoT定稿 | 896 | DoT final translation decisions |
|
||||
| DoT初步 | 412 | DoT preliminary queries |
|
||||
| 偈颂经文名言 | 263 | Verses and sutra quotes |
|
||||
| 成语俗语 | 184 | Idioms and common expressions |
|
||||
| 经论名 | 89 | Sutra/shastra titles |
|
||||
| 内部特色词 | 87 | MPI internal terminology |
|
||||
| 海内外建筑名称 | 28+9 | MPI building/place names |
|
||||
| MPI组织架构 | 4+28 | MPI org structure |
|
||||
| 导师金句 | 24 | Teacher quotes |
|
||||
| 静心学堂课程 | 17+20 | Course names |
|
||||
| 禅意项目 | 14+11 | Zen program terms |
|
||||
| 公案 | 8 | Chan koans |
|
||||
|
||||
## Direct DuckDB
|
||||
|
||||
```
|
||||
duckdb $MPI_PROJECT_ROOT/terms-search/termlib.duckdb
|
||||
```
|
||||
|
||||
Key tables: `unified_terms_flat` (zh, en, loc, source), individual source tables, `unified_terms` view.
|
||||
|
||||
## Rebuilding
|
||||
|
||||
Terms data comes from `$MPI_PROJECT_ROOT/guide/03 术语库/`. To rebuild:
|
||||
1. Convert source xlsx/ods → CSV+YAML in `_output/`
|
||||
2. Rebuild DuckDB from CSVs
|
||||
3. Materialize `unified_terms_flat` view → table for performance
|
||||
|
||||
**Full rebuild pipeline:** See `references/termbase-rebuild.md` (absorbed from the `termbase-management` skill).
|
||||
|
||||
## Translation Alignment
|
||||
|
||||
When aligning translated djot files against the term database, load `references/translation-alignment.md` for the full workflow. Summary:
|
||||
1. Extract Chinese terms from `{% "..." %}` glossary blocks in the translated file
|
||||
2. Batch-search via CLI (`./search.py <term>`). Query each term individually.
|
||||
3. Prioritize DoT定稿 > 内部特色词 > 佛教术语
|
||||
4. Fix both glossary comments AND body-text occurrences
|
||||
5. Verify with grep
|
||||
@@ -1,127 +0,0 @@
|
||||
# Termbase Rebuild (from absorbed termbase-management)
|
||||
|
||||
How to rebuild the terms DuckDB from source spreadsheets. This is the full pipeline from the now-archived `termbase-management` skill.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
```bash
|
||||
pip install openpyxl odfpy pyyaml duckdb
|
||||
```
|
||||
|
||||
## Step 1: Inspect spreadsheet structure
|
||||
|
||||
```python
|
||||
from openpyxl import load_workbook
|
||||
wb = load_workbook(path, read_only=True, data_only=True)
|
||||
for sn in wb.sheetnames:
|
||||
ws = wb[sn]
|
||||
rows = [list(r) for r in ws.iter_rows(min_row=1, max_row=6, values_only=True)]
|
||||
print(f"[{sn}] {sum(1 for _ in ws.iter_rows())} rows, cols: {len(rows[0]) if rows else 0}")
|
||||
for r in rows[:5]: print(f" {r}")
|
||||
```
|
||||
|
||||
For ODS files, use odfpy:
|
||||
```python
|
||||
from odf.opendocument import load as odf_load
|
||||
from odf.table import Table, TableRow, TableCell
|
||||
from odf.text import P
|
||||
|
||||
doc = odf_load(path)
|
||||
for table in doc.getElementsByType(Table):
|
||||
for row in table.getElementsByType(TableRow):
|
||||
cells = row.getElementsByType(TableCell)
|
||||
vals = []
|
||||
for cell in cells:
|
||||
text = ''
|
||||
for p in cell.getElementsByType(P):
|
||||
for node in p.childNodes:
|
||||
if node.nodeType == node.TEXT_NODE:
|
||||
text += node.data
|
||||
vals.append(text.strip() if text else None)
|
||||
```
|
||||
|
||||
## Step 2: Convert to CSV + YAML
|
||||
|
||||
### Cleaning
|
||||
- Strip trailing None/empty values from each row: `while row and not row[-1]: row.pop()`
|
||||
- Skip entirely empty rows
|
||||
- Pad all rows to the max column count
|
||||
|
||||
### Duplicate header handling
|
||||
Some sheets have duplicate column names (e.g., two `英文` columns in paired layout). Deduplicate with suffixes:
|
||||
```python
|
||||
from collections import Counter
|
||||
def dedup_headers(headers):
|
||||
seen = Counter()
|
||||
result = []
|
||||
for h in headers:
|
||||
s = str(h) if h else ''
|
||||
if s in seen:
|
||||
seen[s] += 1
|
||||
result.append(f"{s}_{seen[s]}")
|
||||
else:
|
||||
seen[s] = 1
|
||||
result.append(s)
|
||||
return result
|
||||
```
|
||||
|
||||
### Output formats
|
||||
- **CSV**: `csv.writer` — column-major, preserves all raw data
|
||||
- **YAML**: `yaml.dump(data, allow_unicode=True, default_flow_style=False, sort_keys=False, width=200)` — list of dicts
|
||||
|
||||
## Step 3: Load into DuckDB
|
||||
|
||||
```python
|
||||
import duckdb
|
||||
con = duckdb.connect('termlib.duckdb')
|
||||
|
||||
# Simple CSVs work with auto-detect:
|
||||
con.execute("""
|
||||
CREATE TABLE table_name AS
|
||||
SELECT * FROM read_csv_auto('file.csv', header=true, all_varchar=true)
|
||||
""")
|
||||
```
|
||||
|
||||
### Pitfall: Multiline CSV fields
|
||||
CSV files with embedded newlines (common in glossary example-sentence columns) break DuckDB's auto-sniffer. Fall back to Python csv.reader:
|
||||
|
||||
```python
|
||||
import csv
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
rows = list(csv.reader(f))
|
||||
|
||||
headers = rows[0]
|
||||
data = rows[1:]
|
||||
|
||||
col_defs = ', '.join(f'"{h}" VARCHAR' for h in cleaned_headers)
|
||||
con.execute(f'CREATE TABLE "{table}" ({col_defs})')
|
||||
|
||||
batch_size = 500
|
||||
for i in range(0, len(data), batch_size):
|
||||
batch = data[i:i+batch_size]
|
||||
placeholders = ', '.join(['(' + ', '.join(['?' for _ in headers]) + ')' for _ in batch])
|
||||
flat = [v for row in batch for v in row]
|
||||
con.execute(f'INSERT INTO "{table}" VALUES {placeholders}', flat)
|
||||
```
|
||||
|
||||
### Pitfall: DuckDB CLI opens in-memory by default
|
||||
Running plain `duckdb` gives an empty database. Always pass the file path:
|
||||
```
|
||||
duckdb path/to/termlib.duckdb
|
||||
```
|
||||
|
||||
## Step 4: Create unified views
|
||||
|
||||
See `references/unified-view.sql` for the pattern. Key patterns:
|
||||
- `UNION ALL` across all source tables
|
||||
- Normalize column names to `zh`, `en`, `loc` (出处), `source`
|
||||
- For paired-column sheets (e.g., `中文/英文` + `补充内容/英文_1`), emit two UNION branches
|
||||
- Filter out rows where zh or en is NULL/empty
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **ODS reading**: Must traverse `odf.text.P` child elements, not direct text nodes
|
||||
- **Duplicate headers**: JSON/YAML dict silently overwrites duplicate keys — always deduplicate
|
||||
- **Multiline CSV + DuckDB**: `read_csv_auto` fails on CSVs with quoted newlines — use Python csv.reader
|
||||
- **DuckDB path**: Always explicit file path; `duckdb` alone is in-memory
|
||||
- **`execute_code` sandbox**: Does NOT share pip-installed packages — use `terminal` for Python scripts
|
||||
@@ -1,58 +0,0 @@
|
||||
# Translation Alignment Workflow
|
||||
|
||||
How to align translated djot files against the MPI terms database.
|
||||
|
||||
## When
|
||||
|
||||
After producing a first-pass translation, or when the user asks to check terminology. Any time a `.dj` file contains glossary-style `{% "..." %}` blocks with Chinese→English term pairs.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Read the full translated file. Extract all Chinese terms from `{% "TERM" (pinyin) = ENGLISH ... %}` blocks.
|
||||
|
||||
2. Start the search server: `python3 $MPI_PROJECT_ROOT/terms-search/server.py &` (port 8910). It may already be running — check with `curl -s http://localhost:8910/`.
|
||||
|
||||
3. Batch-search each term via the HTTP API:
|
||||
```
|
||||
curl -s "http://localhost:8910/search?q=TERM&limit=5"
|
||||
```
|
||||
Prefer `src=DoT定稿` filter for authoritative hits, but also check without filter to catch 内部特色词 and 佛教术语 entries.
|
||||
|
||||
4. For each term, compare the database `en` against the file's translation. A mismatch exists when the core term translation differs (ignore explanatory commentary in glossary blocks).
|
||||
|
||||
5. Priority order for which source to trust:
|
||||
- DoT定稿 (highest authority — final translation decisions)
|
||||
- 内部特色词 (MPI internal terminology)
|
||||
- 佛教术语 (general Buddhist terms)
|
||||
- 经论名 (sutra/shastra titles)
|
||||
|
||||
6. Apply fixes with `patch` tool. Fix BOTH the glossary comment AND all body-text occurrences. Use `replace_all=true` for terms that appear identically in multiple places.
|
||||
|
||||
7. After fixing, grep for remaining old forms to verify nothing was missed.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Glossary blocks often include commentary after the term (pinyin, explanations). Compare only the core term translation, not the full comment.
|
||||
- Some terms appear in body text without glossary blocks — scan the body for domain terms too.
|
||||
- The search.py CLI does NOT support `src:` or `loc:` filter syntax directly; use the HTTP API or direct DuckDB queries instead.
|
||||
- Replace-all can create double articles ("the The Eight Steps...") when body text already has the article before the term. Check each replacement site.
|
||||
- Escaped quotes in patch old_string/new_string cause false "Escape-drift" errors. Use unescaped `"` characters from the actual file content.
|
||||
- Terms may have different translations in different contexts (e.g., standalone 心灯 = "lamp of awakening" vs compound 点亮心灯 = "illuminate one's heart"). Use the standalone form for glossary entries.
|
||||
- Sutra quote conventions (e.g., Diamond Sutra "lives" not "bodies" for 身体布施) are not always in the database. Apply standard English Buddhist idiom.
|
||||
|
||||
## Example
|
||||
|
||||
Searching 三无漏学:
|
||||
```
|
||||
curl -s "http://localhost:8910/search?q=三无漏学&limit=5"
|
||||
→ DoT定稿: "three uncontaminated forms of training"
|
||||
→ Current file: "the three undefiled studies"
|
||||
→ MISMATCH → fix
|
||||
```
|
||||
|
||||
## Report
|
||||
|
||||
After alignment, the user may ask for a report. Save it as `alignment-report.dj` in the project directory with:
|
||||
- Summary line (source breakdown)
|
||||
- Per-term before/after table with source annotation
|
||||
- "Not Changed" section listing terms checked and found acceptable
|
||||
@@ -1,132 +0,0 @@
|
||||
# Buddhist Text Translation — Terminology
|
||||
|
||||
Terms encountered in Chinese-English translation of Dharma study materials. These may vary by translator/context; document actual usage per-project.
|
||||
|
||||
## Section headers (common triad)
|
||||
|
||||
| Chinese | English options seen | Notes |
|
||||
|---------|---------------------|-------|
|
||||
| 法义 | Understanding, Dharma Teachings | |
|
||||
| 思考 | Contemplation, Reflection | Consistency within a document matters more than which word |
|
||||
| 练习 | Practice, Application, Exercises | "Application" seen as section header; "Practice"/"Exercise" in running text |
|
||||
|
||||
## Key Buddhist terms
|
||||
|
||||
| Chinese | English | Pitfalls |
|
||||
|---------|---------|----------|
|
||||
| 慈经 | Metta Sutta (Karaniya Metta Sutta) | NOT "Mettavihari Sutta" |
|
||||
| 回向 | Dedication (of merit) | |
|
||||
| 因缘之网 | Web of Causes and Conditions | Also "Network of Causes and Conditions" |
|
||||
| 感恩 | Gratitude | |
|
||||
| 众生 | sentient beings | Consistent throughout |
|
||||
| 使人内心调柔 | makes one's heart gentle | 使人 = makes ONE(self), never "makes others" |
|
||||
| 利益思维 | benefit-oriented thinking | NOT "mindset of benefiting others" — it's about considering benefits TO oneself |
|
||||
| 恩田 | field of gratitude / gratitude as a field of merit | |
|
||||
| 观照 | attend to the mind / mindful observation | Contemplative practice, not intellectual study. NOT "observe" (passive) or "study" (analytical). |
|
||||
| 闻思修 | hearing, contemplating, cultivating | 修 = broad cultivation/practice, not specifically 禅 (meditation). Distinct from 禅修 (meditative cultivation). |
|
||||
| 八步三禅 | Eight Steps and Three Meditations | Community-specific structured contemplative method |
|
||||
| 传帮带 | transmit, help, guide (three-part mentoring) | Core community methodology |
|
||||
| 分灯 | lamp-dividing (decentralization) | Deliberate decentralization of authority across nodes |
|
||||
| 自觉 | self-awareness, voluntary commitment | First of the "Three Spirits" |
|
||||
| 法治 | rule-based governance | Second of the "Three Spirits" |
|
||||
| 无我利他 | selfless service to others | Third of the "Three Spirits" |
|
||||
| 凡夫心 | ordinary mind | Mind governed by afflictions, contrasted with awakened mind |
|
||||
| 贪嗔痴 | greed, anger, ignorance (三毒) | The three root poisons: rāga, dveṣa, moha |
|
||||
| 愿心 | mind of vows, bodhicitta aspiration | Plural "vows" in English |
|
||||
| 重要感、优越感、主宰欲 | sense of importance, superiority, desire to control | Three ego-driven motivations |
|
||||
|
||||
## Structural patterns
|
||||
|
||||
- Section numbering: Chinese uses 一、二、三... English should pick one style (Part One/Two, First/Second, I/II) and stick with it.
|
||||
- Poetry/prayer blocks: Chinese uses parallel structures (感恩... 感恩...; 愿... 愿...). English must match the parallelism.
|
||||
- 愿 (yuàn) at sentence start = optative "May..." — not "we hope that", not "we should".
|
||||
|
||||
|
||||
## Terms from 人生百问 review
|
||||
|
||||
Terms encountered in the *人生百问* review that deserve explicit guidance.
|
||||
These supplement the general table above.
|
||||
|
||||
| Chinese | English | Pitfalls / notes |
|
||||
|---|---|---|
|
||||
| 皈依三宝 | take refuge in the Three Jewels | DoT定稿 preferred. Avoid "take refuge hastily" — use "take refuge lightly" |
|
||||
| 三级修学 | Three-Stage Practice | DoT定稿. Not "Three-Level Study Program" |
|
||||
| 同喜班 | Tongxi Class | MPI convention; verify if glossary requires translation |
|
||||
| 人生佛教 | Buddhism for Human Life | Do not conflate with 人间佛教 / Humanistic Buddhism |
|
||||
| 人间佛教 | Humanistic Buddhism | Distinct from 人生佛教 |
|
||||
| 善知识 | wise teacher / great teacher / authentic teacher | Avoid redundant "great wise teachers" |
|
||||
| 大善知识 | great teacher / wise teacher | Redundant to render as "great wise teacher" |
|
||||
| 依止 | reliance / rely on | "reliance relationship" is awkward; prefer "teacher-student relationship" or "rely on" |
|
||||
| 报身 | reward body / enjoyment body | Verify target convention; MPI often prefers "reward body" (Skt. *sambhogakāya*) |
|
||||
| 化身 | transformation body / emanation body | Verify target convention |
|
||||
| 等流果 | result of equal outflow / correlative effect | Avoid ad-hoc "continuative result" |
|
||||
| 末法 | Dharma-ending age / Age of Dharma Decline | Check terms DB; avoid new coinages |
|
||||
| 暇满 | precious human life / leisure and endowment | "Well-endowed Human Form" is formal |
|
||||
| 暇满人身 | precious human body / well-endowed human form | Context-dependent |
|
||||
| 定课 | daily practice / fixed chanting session / daily recitation | Context-dependent; check terms DB |
|
||||
| 八步骤三种禅修 | Eight Steps and Three Types of Meditation | DoT定稿. Not "Three Kinds of Meditation" |
|
||||
| 八步三禅 | Eight Steps and Three Meditations | Shorter form; verify convention |
|
||||
| 功德 | merit / virtue | Context-dependent; do not default to "merit and virtue" |
|
||||
| 福田 | field of merit / field of blessing | Standard |
|
||||
| 止观 | śamatha-vipaśyanā / calm abiding and special insight | In lists: "śamatha-vipaśyanā" |
|
||||
| 皈依 | refuge / take refuge | In catalogues: noun form "refuge" |
|
||||
| 发心 | bodhicitta / aspiration / arousing the mind | In catalogues: noun form "aspiration" |
|
||||
| 戒律 | precepts / moral discipline | In catalogues: noun form "precepts" |
|
||||
| 正见 | right view | In catalogues: noun form "right view" |
|
||||
| 五戒 | five precepts | Standard |
|
||||
| 惑业 | delusion and karma | Not "confusion and karma" |
|
||||
| 大自在 | great freedom and ease | "Great freedom" alone loses the "ease" nuance |
|
||||
| 开启 | awaken / bring forth / unfold | Not "opened up" |
|
||||
| 迷惑 | delusion | In Buddhist contexts, not "confusion" |
|
||||
| 凡夫心 | mind of an ordinary being | Avoid Chan "ordinary mind" ambiguity |
|
||||
| 加持 | blessing / empowerment | "empowers one another" loses nuance; "supports and blesses one another" |
|
||||
| 四力 (忏悔) | four powers (of confession) | Standard: remorse, support, restraint, refuge/reliance |
|
||||
| 无缘大慈 | unconditional great compassion | Standard |
|
||||
| 同体大悲 | great sympathy of seeing others as oneself | Standard |
|
||||
| 人成即佛成 | When a Human Is Perfected, Buddhahood Is Perfected | Standard rendering |
|
||||
| 登地菩萨 | bodhisattva who has attained the grounds | Standard |
|
||||
| 弟子相 | marks of a disciple | Standard |
|
||||
| 自他相换 | exchanging self and others | Standard |
|
||||
| 明心见性 | clear the mind and see the nature | "Clear mind and seeing the nature" is slightly off |
|
||||
| 言语道断，心行处灭 | Where words fall silent and mental activity ceases | "Words are cut off" is too literal |
|
||||
| 无住生心 | arouse the mind without attachment | Standard |
|
||||
| 诸法如实相 | true suchness of all phenomena | Standard |
|
||||
| 颠倒梦想 | inverted dreams | Standard |
|
||||
| 颠倒妄想 | deluded inversion | Standard |
|
||||
| 十二因缘 | Twelve Links of Dependent Origination | Standard |
|
||||
| 无我 | no-self / anatman | Standard |
|
||||
| 我法二执 | attachment to self and dharmas | Standard |
|
||||
| 三性 (遍计所执性 / 依他起性 / 圆成实性) | three natures: falsely conceived, dependent, perfectly realized | Standard |
|
||||
| 末那 | Manas | Standard (Consciousness-Only) |
|
||||
| 阿赖耶 | Alaya | Standard (Consciousness-Only) |
|
||||
| 唯识 | Consciousness-Only | Standard |
|
||||
| 唯识三十论 | Thirty Verses on Consciousness-Only | Standard |
|
||||
| 认识与存在 | Cognition and Existence | Verify if this is a published translation title |
|
||||
| 三果 | third fruit / anāgāmi | Standard |
|
||||
| 现报 / 生报 / 后报 | reward in this life / next life / future life | "present reward, next-life reward, later reward" is acceptable but less standard |
|
||||
| 中观 | Madhyamaka | Standard |
|
||||
| 天台 | Tiantai | Standard |
|
||||
| 一念三千 | three thousand realms in a single thought | Standard (Tiantai) |
|
||||
| 三传 | Theravada / Chinese Buddhism / Tibetan Buddhism | Standard |
|
||||
| 南传 | Theravada | Standard |
|
||||
| 汉传 | Chinese Buddhism | Standard |
|
||||
| 藏传 | Tibetan Buddhism | Standard |
|
||||
| 佛陀出世的本怀 | Buddha's original purpose in appearing in the world | Standard |
|
||||
| 契理契机 | accordant with principle and adapted to the times | Standard |
|
||||
| 正本清源 | return to the source and clarify what is truly central | Standard |
|
||||
| 随缘 | responding to conditions / in harmony with conditions | Context-dependent |
|
||||
| 随喜 | rejoice | Standard |
|
||||
| 善知识 | wise teacher / good teacher / authentic teacher | Avoid "great wise teachers" |
|
||||
| 定课 | daily practice / fixed chanting session | Context-dependent |
|
||||
|
||||
{% The following terms already appear in the "Key Buddhist terms" table above; they are retained here only when the 人生百问 review adds distinct guidance. %}
|
||||
|
||||
| 法义 | Dharma teachings / understanding | Consistency within a document matters |
|
||||
| 闻思修 | hearing, contemplating, cultivating | 修 is broad cultivation, not just meditation |
|
||||
| 传帮带 | transmit, help, guide | Three-part mentoring |
|
||||
| 分灯 | lamp-dividing | Decentralization of authority |
|
||||
| 自觉 | self-awareness / voluntary commitment | First of the "Three Spirits" |
|
||||
| 法治 | rule-based governance | Second of the "Three Spirits" |
|
||||
| 无我利他 | selfless service to others | Third of the "Three Spirits" |
|
||||
| 重要感、优越感、主宰欲 | sense of importance, superiority, desire to control | Three ego-driven motivations |
|
||||
|
||||
@@ -1,174 +0,0 @@
|
||||
# Common Translation Issues Taxonomy
|
||||
|
||||
From 译文常见问题与案例示范 (Common Issues in Translation, 2026-03-30).
|
||||
Categorized by accuracy and readability.
|
||||
|
||||
## Accuracy Issues
|
||||
|
||||
Standard: convey meaning at the content level, not word-by-word correspondence.
|
||||
"Translation is not simply word conversion."
|
||||
"You must understand what thought it is actually trying to express."
|
||||
|
||||
### 1. Omission (漏翻)
|
||||
|
||||
Content present in source but absent from target.
|
||||
|
||||
Example:
|
||||
> 随着黑白牌的运用，象征这部分已能为大家提供系统的学习和训练资料。
|
||||
>
|
||||
> With the help of the Black-and-White Cards, this approach now offers a structured
|
||||
> system of study.
|
||||
>
|
||||
> *Missing:* 学习和训练 (learning and training) — "study" only covers half.
|
||||
> Also: 象征这部分 (symbolizing this aspect) is dropped.
|
||||
|
||||
### 2. Mistranslation (错翻)
|
||||
|
||||
#### a. Over-free translation, weakening the original
|
||||
|
||||
> "道德"一词，人们耳熟能详。即使在道德日渐边缘化的今天，人们依然会用"这人
|
||||
> 很有道德""这么做太缺德了"来评价周遭人事。
|
||||
>
|
||||
> "Morality" is a familiar term, and even in today's world, where it has become
|
||||
> increasingly marginalized, expressions like "immorality" are still commonly used
|
||||
> to describe certain behaviors and individuals.
|
||||
>
|
||||
> *Issue:* The source gives two concrete colloquial expressions ("这人很有道德" /
|
||||
> "这么做太缺德了") — the target collapses them into a single abstract "expressions
|
||||
> like 'immorality'." The vividness and specificity are lost.
|
||||
|
||||
#### b. Over-literal translation, harming comprehension
|
||||
|
||||
> 作为素菜馆，应该尽量提供绿色食品。
|
||||
>
|
||||
> A vegetarian restaurant should strive to offer green food.
|
||||
>
|
||||
> *Issue:* 绿色食品 = "safe and healthy food" (Chinese idiom), not
|
||||
> "green food" (color of food in English). → `safe and healthy options`
|
||||
|
||||
#### c. Misunderstanding the source
|
||||
|
||||
> 政府现在倡导文化自信，习主席提出的"讲仁爱，重民本，守诚信，崇正义，尚和合，求大同"。
|
||||
>
|
||||
> President Xi Jinping has called for a confident embrace of Chinese culture,
|
||||
> highlighting virtues such as...
|
||||
>
|
||||
> *Issue:* Two different subjects: the government advocates cultural confidence;
|
||||
> Xi proposed the six virtues. The target merges both into Xi alone.
|
||||
|
||||
#### d. Wrong word choice causing misunderstanding
|
||||
|
||||
> 提供一些通俗易懂的法宝
|
||||
>
|
||||
> with easy Buddhist materials
|
||||
>
|
||||
> *Issue:* 通俗易懂 means "accessible to a general audience," not "easy/simple."
|
||||
> The Dharma can be accessible but is never "simple." → `accessible`
|
||||
|
||||
### 3. Overtranslation (多翻)
|
||||
|
||||
Adding parentheticals or expansions not in the source.
|
||||
|
||||
> 出家人不仅要对三宝、师长、道友建立没有染污的情感
|
||||
>
|
||||
> Monastics should cultivate pure and untainted emotions not only toward the Three
|
||||
> Jewels (the Buddha, the Dharma, the Sangha), their teachers (The Head Monk and
|
||||
> Abbot), and fellow practitioners...
|
||||
>
|
||||
> *Issue:* Parenthetical expansions like "(the Buddha, the Dharma, the Sangha)"
|
||||
> and "(The Head Monk and Abbot)" are not in the source and should be removed.
|
||||
|
||||
### 4. Terminology Errors
|
||||
|
||||
- 身心 → `body and heart mind` → should be `body and mind`
|
||||
- 甘露别院 → `Ganlu Bieyuan The Amrita Retreat Center` → official: `Amrita Retreat Center`
|
||||
- "六字" → `six characters` → should be `six words`
|
||||
|
||||
### 5. Detail-Level Issues
|
||||
|
||||
- Tone appropriateness
|
||||
- Citation source authority
|
||||
- CN→EN conversion rules: numerals, units, date formats
|
||||
- Proper noun handling: transliteration vs. translation
|
||||
|
||||
## Readability Issues
|
||||
|
||||
Standard: fluent sentences, graceful expression, matching target reader habits.
|
||||
"Conform to what foreign readers find readable."
|
||||
"Concise, clear, accurate, not draggy, meaning clear."
|
||||
|
||||
### 1. Redundancy & Wordiness
|
||||
|
||||
#### a. Long / nested sentences
|
||||
|
||||
Sentences with too many clauses that can't be read in one breath. Split them.
|
||||
|
||||
#### b. Passive voice overuse
|
||||
|
||||
> 如果对肉食消费得少，乃至完全不消费，那么从业者自然随之减少，很多动物就可以摆脱被屠宰、割截的厄运。
|
||||
>
|
||||
> If meat consumption were reduced, or even eliminated entirely, then the number of
|
||||
> businesses involved would naturally be lessened. This way, many animals could be
|
||||
> spared from the adversity of being slaughtered and butchered.
|
||||
>
|
||||
> *Issue:* Three passives in two sentences. Active rewrite is more concise:
|
||||
> *If people consume less meat — or none at all — fewer will work in the industry,
|
||||
> and many animals will be spared the fate of slaughter and dismemberment.*
|
||||
|
||||
#### c. Nominalization (动词名词化)
|
||||
|
||||
> 我们前期倡导的正念禅修，比较重视培养觉知。
|
||||
>
|
||||
> In the early stages of mindfulness practice, we placed greater emphasis on
|
||||
> cultivating awareness.
|
||||
>
|
||||
> *Issue:* "placed greater emphasis on" is nominalized. → `we emphasized`
|
||||
|
||||
### 2. Poor Structure (结构不当)
|
||||
|
||||
Top-heavy sentences — the main clause arrives too late.
|
||||
|
||||
> 没有如法的生活，不能严格要求自己，在今天这个红尘滚滚的时代，我们简直是没希望的。
|
||||
>
|
||||
> Weathering today's deluge of worldly temptation, without a Dharma-aligned lifestyle
|
||||
> and strict self-discipline, we have no hope.
|
||||
>
|
||||
> *Issue:* The condition clauses pile up before the main point. Restructure:
|
||||
> *Without a Dharma-aligned lifestyle and strict self-discipline, we have no hope
|
||||
> of weathering today's deluge of worldly temptations.*
|
||||
|
||||
### 3. Wrong Word Choice (用词不当)
|
||||
|
||||
> 当一个人开始关心生命的大问题时，对小问题自然云淡风轻。
|
||||
>
|
||||
> when we begin to contemplate the profound questions of existence, the banal
|
||||
> iterations of daily life lose their significance.
|
||||
>
|
||||
> *Issue:* "banal iterations" is obscure. Simplify.
|
||||
|
||||
### 4. Weak Flow & Transitions (流畅性 & 连接词)
|
||||
|
||||
> 修行的核心就是止恶行善。为什么某种行为能成为生命的主导？就是因为不断重复。
|
||||
>
|
||||
> The essence of cultivation lies in ceasing unwholesome actions and promoting
|
||||
> wholesome ones. But why do certain behaviors dominate our lives? Because of
|
||||
> constant repetition.
|
||||
>
|
||||
> *Issue:* Adding transition words improves flow. The "But" here creates a false
|
||||
> contrast — the second sentence is explanation, not counterpoint.
|
||||
|
||||
## Detection Checklist
|
||||
|
||||
When reviewing, run through:
|
||||
- [ ] Any missing content? (compare paragraph-by-paragraph)
|
||||
- [ ] Any added content? (parentheticals, expansions not in source)
|
||||
- [ ] Any over-literal renderings that don't work in English? (idioms, set phrases)
|
||||
- [ ] Any over-free renderings that lose specificity? (concrete examples → abstract)
|
||||
- [ ] Subject confusion? (two actors merged into one)
|
||||
- [ ] Buddhist terminology checked against terms DB?
|
||||
- [ ] Sentences too long to read in one breath? (split at natural breaks)
|
||||
- [ ] Passive voice clustering? (3+ in a paragraph)
|
||||
- [ ] Nominalized verbs? ("placed emphasis on" → "emphasized")
|
||||
- [ ] Top-heavy sentence structure? (main clause buried after long preamble)
|
||||
- [ ] Obscure word choices? (would a general reader understand?)
|
||||
- [ ] Missing or misleading transition words?
|
||||
@@ -1,90 +0,0 @@
|
||||
# Deliberation Protocol (审议规程)
|
||||
|
||||
From 东方译场审议手册 (Oriental Translation Review Manual, 2026-03-25).
|
||||
Guidance from Ven. Jiqun on translation review culture and process.
|
||||
|
||||
## Core Standards
|
||||
|
||||
1. **Accuracy (准确性)** — primary. Convey content-level meaning, not word-by-word
|
||||
correspondence. Deepen accuracy iteratively; grasp the "inner spirit" rather than
|
||||
fixating on literal vs free translation.
|
||||
|
||||
2. **Readability (可读性)** — secondary. Fluent sentences, graceful expression. Do
|
||||
not pursue endless revision; what is achievable now is the best for now.
|
||||
|
||||
**Test**: invite 5–10 readers to read aloud and give feedback — is it clear or
|
||||
obscure? Is it concise? Is the meaning clear?
|
||||
|
||||
## Decision-Making
|
||||
|
||||
When opinions differ:
|
||||
- Compare which version is more accurate and more readable
|
||||
- Do NOT cling to your own view (不执着于己见)
|
||||
- If two people cannot decide, invite 5–10 to read and vote
|
||||
- Remember: the translation is for the READERS, not for yourself
|
||||
|
||||
## Core Principle: Rejoice First (随喜)
|
||||
|
||||
Before flagging issues, affirm what is good. Professional translation companies
|
||||
emphasize positive feedback to build translator confidence and good collaboration.
|
||||
Criticism alone — however precise — breeds resistance and harms the final result.
|
||||
|
||||
In the Oriental Translation Workshop, rejoicing in others' merits cultivates
|
||||
sympathetic joy (随喜), compassion, selflessness, and boundless merit — while
|
||||
encouraging others' wholesome efforts.
|
||||
|
||||
**Practice**: develop eyes that SEE the merits in a translation. Rejoice with
|
||||
genuine gladness. Examples:
|
||||
- "So many technical terms and complex logic here — you handled it beautifully!"
|
||||
- "This word choice is exactly right."
|
||||
- "The prose is so clean and fluid!"
|
||||
- "Excellent!" / "Good word choice!"
|
||||
|
||||
## Three Principles for Giving Suggestions
|
||||
|
||||
1. **Lower the self** — use compassionate, gentle, skillful language
|
||||
2. **Be diligent and attentive** — find the actual problems
|
||||
3. **Tier your suggestions** — make clear what MUST change, what COULD improve,
|
||||
and what needs DISCUSSION
|
||||
|
||||
## Three-Tier Issue System
|
||||
|
||||
### Level 1: Spelling, Grammar, Format (拼写、语法、格式)
|
||||
|
||||
**Action**: fix directly, keep tracked changes visible.
|
||||
|
||||
May annotate for translator development:
|
||||
- "MPI format uses American English — `toward` not `towards`."
|
||||
- "Use `Chan` here, not `Zen`."
|
||||
- "MPI format does not use diacritics — `Yogacara` not `Yogācāra`."
|
||||
|
||||
### Level 2: Omission, Mistranslation, Wordiness, Tone, Word Choice
|
||||
(漏翻、错翻、啰嗦、语气问题、用词问题)
|
||||
|
||||
**Action**: preferably flag for translator to fix themselves; may also fix
|
||||
directly with tracked changes.
|
||||
|
||||
Suggested language:
|
||||
- "Did we miss something here?"
|
||||
- "This sentence is quite long — could we split it into 2–3?"
|
||||
- "The translation here differs from the original. It reads as if the teacher
|
||||
is saying X, but I think he means Y. What do you think?"
|
||||
- "This word feels a bit strong here — your thoughts?"
|
||||
- (After direct adjustment) "I tightened this sentence — does this work?"
|
||||
|
||||
### Level 3: Citation Versions, Slightly Awkward Wording, Marginal Readability
|
||||
(引用版本不当、用词稍微不当、阅读体验略差)
|
||||
|
||||
**Action**: flag and discuss with translator.
|
||||
|
||||
Suggested language:
|
||||
- "I found another citation version that may be more concise — which do you prefer?"
|
||||
- "Could we use this word instead? Might fit better."
|
||||
- "Could you take another look at this passage? I feel readability could improve
|
||||
but I'm not sure how — your thoughts?"
|
||||
|
||||
## Tone Guide
|
||||
|
||||
- Address the translator as 菩萨 (Bodhisattva) — respectful peer address
|
||||
- Use questions, not commands: "Could we...?" "What do you think?" "Would this work?"
|
||||
- Frame as collaborative inquiry, not correction
|
||||
@@ -1,171 +0,0 @@
|
||||
# Extracting `.docx.md` to `source.dj` + `bilingual.dj`
|
||||
|
||||
When the input is a `.docx.md` file (already pandoc markdown, not plain text), the
|
||||
extraction differs from the standard DOCX→plain workflow. The markdown preserves
|
||||
formatting artifacts that need specific handling.
|
||||
|
||||
## CN/EN boundary detection in merged lines
|
||||
|
||||
Pandoc markdown often merges CN and EN text on heading lines where the DOCX had
|
||||
multiple runs in the same paragraph:
|
||||
|
||||
```
|
||||
# **三、重视文化教育，重塑人生价值** Prioritize Cultural Education; Reshape Life Values {#三、...}
|
||||
```
|
||||
|
||||
The boundary regex must account for whitespace on BOTH sides of `**` markers:
|
||||
|
||||
```python
|
||||
# WRONG: \** then \s* — fails when there's space BEFORE * (值 *Realizing)
|
||||
r'[\u4e00-\u9fff](?:\*{0,2})\s*([A-Za-z])'
|
||||
|
||||
# WRONG: \s* then \** — fails when there's space AFTER ** (值** Prioritize)
|
||||
r'[\u4e00-\u9fff]\s*\**([A-Za-z])'
|
||||
|
||||
# CORRECT: whitespace on both sides of optional *
|
||||
r'[\u4e00-\u9fff]\s*\**\s*([A-Za-z])'
|
||||
```
|
||||
|
||||
Include CJK punctuation ranges in the character class:
|
||||
`[\u4e00-\u9fff\u3000-\u303f\uff00-\uffef]`
|
||||
|
||||
## Anchor stripping
|
||||
|
||||
Strip BOTH `{#anchor}` (pandoc heading anchors) AND `[text](#link)` (markdown TOC links)
|
||||
before further processing:
|
||||
|
||||
```python
|
||||
def strip_anchors(s):
|
||||
s = re.sub(r'\{#[^}]*\}', '', s) # {#anchor}
|
||||
s = re.sub(r'\[([^\]]*)\]\([^)]*\)', r'\1', s) # [text](#link) → text
|
||||
return s
|
||||
```
|
||||
|
||||
## TOC handling — two critical guards
|
||||
|
||||
### Guard 1: Don't pair TOC CN entries with following EN lines
|
||||
|
||||
TOC entries often look like CN-heading-followed-by-EN (the standard pair pattern),
|
||||
but the following EN is actually the next TOC entry or heading:
|
||||
|
||||
```
|
||||
[一、营造禅意氛围，优化工作环境\t1](#anchor)
|
||||
[二、重视慈善关爱...]
|
||||
...
|
||||
1、Create a Chan(Zen) Atmosphere; Optimize Your Environment
|
||||
2、Focus on Compassion and Care; Build Good Relationships
|
||||
```
|
||||
|
||||
If the last CN TOC entry is followed by a blank line then the first EN TOC entry,
|
||||
the "CN → blank → EN" heading pattern will consume the first EN TOC entry.
|
||||
Guard against this:
|
||||
|
||||
```python
|
||||
def is_toc_line(line, cn_raw):
|
||||
if re.search(r'\[.*\]\(.*\)', line): # markdown link
|
||||
return True
|
||||
if re.search(r'\t\d+', cn_raw): # tab + page number
|
||||
return True
|
||||
return False
|
||||
```
|
||||
|
||||
Skip the CN→EN and CN→blank→EN patterns when `is_toc_line()` returns True.
|
||||
These CN TOC entries should stay unpaired (en='') and get their EN from the
|
||||
separate EN TOC list.
|
||||
|
||||
### Guard 2: Pair TOC EN entries forward, not backward
|
||||
|
||||
EN TOC entries appear as standalone non-CJK lines. They must be paired with
|
||||
the FIRST unpaired CN (not the last):
|
||||
|
||||
```python
|
||||
# CORRECT: forward iteration
|
||||
for j in range(len(pairs)):
|
||||
if not pairs[j][1] and has_cjk(pairs[j][0]):
|
||||
pairs[j] = (pairs[j][0], en)
|
||||
break
|
||||
|
||||
# WRONG: reverse iteration (pairs last-first, shifting everything)
|
||||
for j in range(len(pairs)-1, -1, -1):
|
||||
...
|
||||
```
|
||||
|
||||
## Orphaned trailing `*` from italic splits
|
||||
|
||||
When a line has italicized EN text (`*Realizing Ultimate Value*`) and the split
|
||||
point is at the `R` (after the opening `*` is consumed by the CN cleanup), the
|
||||
EN text retains a trailing `*`: `Realizing Ultimate Value*`. Strip it:
|
||||
|
||||
```python
|
||||
def clean_en(s):
|
||||
s = re.sub(r'^\d+[、,.]\s*', '', s)
|
||||
s = re.sub(r'\*+$', '', s) # orphaned italic close
|
||||
return s.strip()
|
||||
```
|
||||
|
||||
## `clean_cn` — operation order matters
|
||||
|
||||
Strip leading numbers BEFORE heading markers. A line like `1. # **营造禅意...`
|
||||
starts with a digit, so `^#+\s*\**` won't match until the number is gone:
|
||||
|
||||
```python
|
||||
def clean_cn(s):
|
||||
s = re.sub(r'^\d+\.\s*', '', s) # leading number FIRST
|
||||
s = re.sub(r'^#+\s*\**', '', s) # then heading markers
|
||||
s = re.sub(r'\**\s*$', '', s) # trailing **
|
||||
s = re.sub(r'\t\d+$', '', s) # trailing page number
|
||||
return s.strip()
|
||||
```
|
||||
|
||||
## Full extraction recipe
|
||||
|
||||
1. Read `.docx.md` text
|
||||
2. For each line: strip anchors, detect CJK/EN
|
||||
3. CJK lines: try `split_cnen()` first (merged CN+EN). If that gives EN, use it.
|
||||
Otherwise look ahead for EN on next line or next+blank — but SKIP if TOC-like.
|
||||
4. EN-only lines: pair forward with first unpaired CN.
|
||||
5. Clean: strip heading markers, leading numbers, trailing page numbers.
|
||||
6. Write `source.dj` (CN only) and `bilingual.dj` with proper markdown structure.
|
||||
|
||||
## bilingual.dj output format
|
||||
|
||||
Must follow the project convention (see reference bilingual.dj in any completed article):
|
||||
|
||||
```
|
||||
# CN Title
|
||||
# EN Title
|
||||
|
||||
---CN Subtitle
|
||||
---EN Subtitle
|
||||
|
||||
CN Author
|
||||
EN Author
|
||||
|
||||
- CN TOC item 1
|
||||
- CN TOC item 2
|
||||
...
|
||||
|
||||
- EN TOC item 1
|
||||
- EN TOC item 2
|
||||
...
|
||||
|
||||
CN body paragraph
|
||||
EN body paragraph
|
||||
|
||||
CN section heading ← plain text, no # prefix
|
||||
EN section heading
|
||||
|
||||
...
|
||||
|
||||
CN sub-heading ← plain text, e.g. "1. 创造精神财富"
|
||||
EN sub-heading
|
||||
```
|
||||
|
||||
- Title: `# ` prefix on both CN and EN
|
||||
- Subtitle: `---` prefix (3 dashes, no space after)
|
||||
- Author: plain text, one pair
|
||||
- TOC: `- ` bullet, CN block then EN block (not interleaved), blank line between blocks
|
||||
- Body headings: plain text, no `#` or `##` markers. Hierarchy conveyed by numbering:
|
||||
`一、二、三、` for major sections, `1. 2. 3.` for sub-sections
|
||||
- Body paragraphs: interleaved (CN line, EN line, blank)
|
||||
- `source.dj` uses `# Title`, `---subtitle`, `## section headings` — different from bilingual.dj which uses plain body headings
|
||||
@@ -1,65 +0,0 @@
|
||||
# Edit Suggestions in bilingual.dj
|
||||
|
||||
When the user asks to add edit suggestions directly into bilingual.dj, use this
|
||||
two-tier approach.
|
||||
|
||||
## Two files
|
||||
|
||||
- `bilingual.dj` — clean: inline corrections applied, NO `{% %}` comment lines
|
||||
- `commented.dj` — same content + `{% ... %}` comment lines interleaved
|
||||
|
||||
This lets the user diff them side by side.
|
||||
|
||||
## Inline corrections
|
||||
|
||||
Apply directly to the English text. These are fixes the reviewer is confident about:
|
||||
|
||||
- Typos, mechanical issues (Chinese punctuation in EN, double periods, stray `*`, unbalanced quotes)
|
||||
- Grammar fixes (subject-verb agreement, missing articles)
|
||||
- Wording improvements (clunky literal translations → idiomatic English)
|
||||
|
||||
Apply with `patch` (mode='replace'). Verify uniqueness before
|
||||
replacing — many EN paragraphs are long single lines, so match a unique
|
||||
substring. Never use regex-based string replacement in `execute_code`.
|
||||
|
||||
## `{% %}` comment lines
|
||||
|
||||
For translation decisions worth documenting but not "correct" per se:
|
||||
|
||||
```
|
||||
CN paragraph
|
||||
EN paragraph
|
||||
{% reason for the choice, alternative renderings %}
|
||||
(blank)
|
||||
```
|
||||
|
||||
The comment goes AFTER the EN line (before the blank separator). One comment per
|
||||
issue. Keep them terse.
|
||||
|
||||
What to comment on:
|
||||
- Literal translation of idioms (`单线程` → "single-threaded")
|
||||
- Translation choices that differ from literal meaning (`精神利益` → "nourishing the spirit")
|
||||
- Standard Buddhist terminology (`正命` → "Right Livelihood")
|
||||
- Glosses added/dropped (`道场` — "(Dojo)" parenthetical removed)
|
||||
- Idiom translations (`甘之若饴` → "as sweet as syrup")
|
||||
- Paired terms where one rendering influences the other (`魔性` → "demon-nature" to parallel "Buddha-nature")
|
||||
- Scripture citations with Sanskrit titles (`普贤行愿品` → "Gaṇḍavyūha Sūtra")
|
||||
|
||||
What NOT to comment on:
|
||||
- Obvious corrections (typos, grammar)
|
||||
- Names, dates, simple connectives
|
||||
- Standard renderings with no interesting choice
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Never split a paragraph mid-sentence** with a `{% %}` comment. Comments go
|
||||
AFTER the full EN paragraph, before the blank separator. If a string
|
||||
replacement inserts `\n{% %}` in the middle of a paragraph, it breaks the
|
||||
interleaved structure.
|
||||
|
||||
- **Check for merged comments**: after inserting, scan for `{% ... %} ` followed
|
||||
by EN text on the same line. These must be split so the EN text continues on
|
||||
the next line.
|
||||
|
||||
- **Collapse triple+ blank lines**: comment insertions can create extra blanks.
|
||||
Run `re.sub(r'\n\n\n+', '\n\n', text)` after all insertions.
|
||||
@@ -1,116 +0,0 @@
|
||||
# Proofreading: Manuscript vs Typeset
|
||||
|
||||
AGENTS.md defines two workflows: Translation (A) and Proofread (B).
|
||||
The workflows below are Proofread mode — English comes from an existing
|
||||
manuscript and is authoritative. Only flag mechanical/manuscript-level issues.
|
||||
|
||||
## Two extraction workflows
|
||||
|
||||
### A. Bilingual from DOCX (standard)
|
||||
|
||||
When the DOCX manuscript has both Chinese and English in 1:1 paragraph
|
||||
correspondence, generate `bilingual.dj` directly from the DOCX:
|
||||
|
||||
1. `pandoc docx → plain text`
|
||||
2. Extract Chinese-English pairs from body
|
||||
3. Apply fixes: italicize Sanskrit on first occurrence, fix `N.Letter` → `N. Letter` spacing
|
||||
4. Write bilingual.dj
|
||||
The DOCX English is the authoritative target text. No PDF needed.
|
||||
|
||||
**Extraction approach**: start by adapting `scripts/gen-bilingual-docx.py`.
|
||||
For articles where the body has strict CN→EN→CN→EN alternation, the simple
|
||||
extraction in that script (CN line, blank, EN line, blank) works directly.
|
||||
|
||||
### A2. Bilingual from `.docx.md` (pandoc markdown output)
|
||||
|
||||
When working with an already-converted `.docx.md` file (pandoc markdown, not plain
|
||||
text), use the techniques in `references/docx-md-extraction.md`. Key differences
|
||||
from plain-text extraction: merged CN+EN on heading lines, `{#anchor}` and
|
||||
`[text](#link)` artifacts, TOC pairing guards.
|
||||
|
||||
### B. Bilingual from PDF (when PDF is the typeset target)
|
||||
sections that order CN content before EN content (CN heading → CN body → EN heading →
|
||||
EN body), the simple alternation fails. Use block-based extraction instead:
|
||||
|
||||
1. Tag each non-blank line as CN or EN (via `has_cjk()`)
|
||||
2. Join page-break split paragraphs: merge consecutive same-language paragraphs
|
||||
only when the first is long (>30 chars), doesn't end with CJK/ASCII terminal
|
||||
punctuation (`[。！？：）.?!]$`), and isn't heading-like (starts with
|
||||
`^[\dIVX]+[\.\s]` and <60 chars)
|
||||
3. Group consecutive same-language items into blocks
|
||||
4. Walk blocks: for each CN block, pair with the next EN block via `zip()`.
|
||||
`min(len(cn), len(en))` handles translator-introduced paragraph splits.
|
||||
|
||||
**Page-break splits in pandoc plain-text output**: the DOCX→plain conversion
|
||||
sometimes splits a Chinese paragraph mid-sentence (e.g. `白居` + `易、苏轼…`).
|
||||
These appear as two consecutive CN lines separated by a blank. The join heuristic
|
||||
above catches these reliably. For EN text, page-break splits are rare; the heading
|
||||
detection (`^[\dIVX]+[\.\s]`, <60 chars) prevents false merges of EN headings
|
||||
with following EN body paragraphs.
|
||||
The DOCX English is the authoritative target text. No PDF needed.
|
||||
|
||||
### B. Bilingual from PDF (when PDF is the typeset target)
|
||||
|
||||
When the PDF English is the typeset "final" version and should be the target:
|
||||
|
||||
1. Extract DOCX Chinese paragraphs (source)
|
||||
2. Extract PDF body text via `pdftotext -layout`
|
||||
3. Clean PDF: remove slug lines, headers, page numbers, join hyphenation breaks
|
||||
4. Match DOCX English paragraphs against PDF body to find positions
|
||||
5. Segment PDF body at matched positions
|
||||
6. Write bilingual.dj with Chinese source + PDF English target
|
||||
|
||||
**Pitfalls in PDF extraction:**
|
||||
- Consecutive hyphenation breaks (e.g. `thou-` + `sand...al-` + `leviate`) — the join
|
||||
loop must be recursive: after joining pair N, check if result still ends with `-`
|
||||
and join with line N+2
|
||||
- Lines with leading whitespace: use `lstrip()` before checking `n[0].islower()`
|
||||
- Drop-cap artifacts: `L iving` → `Living`
|
||||
- Trailing section numbers: `...viewpoints. 1)` — the ` 1)` is a PDF section marker
|
||||
bleeding into the previous paragraph
|
||||
|
||||
### C. Edit suggestions (edit-suggestions.dj)
|
||||
|
||||
After generating bilingual.dj, scan for issues and write `edit-suggestions.dj`:
|
||||
|
||||
**Format**: follow the original document's section/chapter layout. Group suggestions
|
||||
under the chapter headings where the issues occur. Use diff-style `-/+` notation.
|
||||
|
||||
**What to flag:**
|
||||
- Garbled Chinese text (merged duplicate edits in source DOCX)
|
||||
- Repeated words (`the The`)
|
||||
- Chapter numbering mismatches (e.g. `九` ↔ `VIII`)
|
||||
- Translator notes in headings (`（某某翻，某某审）`)
|
||||
- Missing quotes around dialogue/speech
|
||||
|
||||
## Common source DOCX issues
|
||||
|
||||
- Translator notes in Chinese headings: `（某某翻，某某审）` — delete for publication
|
||||
- Merged duplicate edits: cut-paste errors where old+new text appear together
|
||||
- `N.Letter` without space: `2.How` → `2. How`
|
||||
- `the The` double article
|
||||
- **Numbering mismatches**: CN and EN headings sometimes disagree (e.g. CN `3．` vs EN `2.`).
|
||||
The TOC usually has the correct number — flag the body heading for correction.
|
||||
- **Doubled names**: `岳麓书院岳麓书院` — cut-paste artifacts in Chinese body text.
|
||||
- **EN paragraph splits without CN counterpart**: translator sometimes renders one CN
|
||||
paragraph as two EN paragraphs. The block-based extractor drops the extra EN paragraph
|
||||
(as `min(len_cn, len_en)`). Flag in edit-suggestions so it can be manually merged or
|
||||
the CN paragraph can be split.
|
||||
|
||||
## Sanskrit italicization
|
||||
|
||||
On first occurrence in body text, wrap with `*term*`. Track seen terms across
|
||||
the full body. Terms: bodhisattva, bodhicitta, samsara, Dharma, karma, nirvana,
|
||||
Sangha, sutra, Mahayana, Sravaka, Vinaya, Lamrim, Ksitigarbha, Samantabhadra,
|
||||
Chan, Arhatship, Theravada.
|
||||
|
||||
## Proofread scope boundary
|
||||
|
||||
When proofreading a DOCX manuscript:
|
||||
- **DO flag**: typos, double words, double punctuation, numbering mismatches,
|
||||
garbled text, translator notes, duplicate names, capitalization errors.
|
||||
- **Do NOT flag**: em-dash formatting (`—` vs `---`), terminology choices,
|
||||
translation style, calques, word order. The manuscript English is authoritative.
|
||||
- **Do NOT apply fixes** — write `edit-suggestions.dj` only.
|
||||
- If the user asks for translation review separately, write findings to
|
||||
`translation-findings.dj`.
|
||||
@@ -1,64 +0,0 @@
|
||||
# Terms Database Alignment
|
||||
|
||||
Batch-align translation glossary entries and body text against the MPI terms database.
|
||||
|
||||
## Module API (preferred)
|
||||
|
||||
Import directly in `execute_code` scripts — no subprocess, no server, no text parsing:
|
||||
|
||||
```python
|
||||
import sys
|
||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
||||
from search import search
|
||||
|
||||
results = search("三级修学", limit=5)
|
||||
results = search("空性", loc="心经", src="DoT定稿", limit=5)
|
||||
# returns list of {zh, en, loc, source} dicts
|
||||
```
|
||||
|
||||
## Batch lookup pattern
|
||||
|
||||
```python
|
||||
import sys
|
||||
sys.path.insert(0, '$MPI_PROJECT_ROOT/terms-search')
|
||||
from search import search
|
||||
|
||||
terms = ["三无漏学", "八步三禅", "闻思修", ...]
|
||||
author_sources = {"DoT定稿", "内部特色词", "佛教术语", "经论名"}
|
||||
|
||||
for term in terms:
|
||||
results = search(term, limit=10)
|
||||
relevant = [r for r in results if r["source"] in author_sources]
|
||||
for r in relevant:
|
||||
print(f"{r['zh']} → {r['en']} [{r['source']}]")
|
||||
```
|
||||
|
||||
Or filter to a single authoritative source directly:
|
||||
```python
|
||||
results = search("三级修学", src="DoT定稿", limit=5)
|
||||
```
|
||||
|
||||
## Priority ranking
|
||||
|
||||
When the same term has entries in multiple source tables, prefer:
|
||||
1. DoT定稿 (highest authority — final translation decisions)
|
||||
2. 内部特色词 (MPI internal terminology)
|
||||
3. 佛教术语 (general Buddhist terminology)
|
||||
4. 经论名 (sutra/shastra titles)
|
||||
|
||||
## Alignment workflow
|
||||
|
||||
1. Extract all Chinese glossary terms from `{% "TERM" ... %}` blocks in the .dj file
|
||||
2. Extract body-text domain terms that may not have glossary entries
|
||||
3. Batch-query each term against the HTTP API
|
||||
4. Filter results to authoritative source tables
|
||||
5. Compare DB canonical translation against current file translation
|
||||
6. Flag mismatches where DB entry differs materially from current
|
||||
7. Apply fixes with `patch` tool — fix both glossary comments AND body text occurrences
|
||||
8. Verify with `grep` that no old terms remain
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- `replace_all` can create doubled words when the surrounding context already contains the replacement string (e.g., "The Eight Steps" → "The The Eight Steps"). Prefer targeted single-replacement patches.
|
||||
- Start patches from the bottom of the file upward to preserve line numbers.
|
||||
- Some DB entries are contextual phrases (e.g., "珍惜法缘" → a full sentence), not standalone term translations. Use standalone term entries where available.
|
||||
@@ -1,441 +0,0 @@
|
||||
# Translation Pitfalls — MPI Buddhist Texts
|
||||
|
||||
Patterns found in CN→EN translation review. Add to this file as new patterns emerge.
|
||||
|
||||
## Terminology conflation
|
||||
|
||||
### 人生佛教/人间佛教 — distinct concepts
|
||||
|
||||
人生佛教 (Taixu's "Buddhism for Human Life") and 人间佛教 (Yinshun's "Humanistic Buddhism")
|
||||
are distinct doctrinal positions in modern Chinese Buddhism. Do not conflate both into
|
||||
"Humanistic Buddhism." When the source uses 人生佛教, render as "Buddhism for Human Life"
|
||||
or "Human Life Buddhism." User may propose a deliberately non-standard rendering
|
||||
("Buddhism for daily lives") — that's their call. Don't unilaterally pick from
|
||||
the standard set without checking.
|
||||
|
||||
### 心性论 → buddha-nature (WRONG)
|
||||
|
||||
心性 (mind-nature) is broader than 佛性 (buddha-nature / tathāgatagarbha).
|
||||
When a text discusses 心性 in the context of Confucian self-cultivation or general
|
||||
Buddhist psychology, use "mind-nature" or "nature of mind." Reserve "buddha-nature"
|
||||
only when the text explicitly references tathāgatagarbha doctrine.
|
||||
|
||||
### 恨 → resentment (WRONG)
|
||||
|
||||
恨 means "hatred," not "resentment." In the triad 羡慕嫉妒恨 (envy, jealousy, hatred),
|
||||
the force is strong. "Resentment" is too mild.
|
||||
|
||||
### 感悟 → conversant / heartfelt (WRONG)
|
||||
|
||||
感悟 means experiential insight or realization. It is not intellectual familiarity
|
||||
("conversant") or emotional warmth ("heartfelt"). Render as "insight," "realization,"
|
||||
or "deep understanding."
|
||||
|
||||
### 关爱/关怀 → compassion (WRONG)
|
||||
|
||||
Chinese 关爱 and 关怀 mean "care" or "loving care." They are NOT 慈悲 (compassion / karuṇā).
|
||||
Conflating them obscures two distinct Buddhist concepts.
|
||||
|
||||
Check every occurrence of "compassion" in a translation against the source:
|
||||
- If source is 关爱/关怀 → "care"
|
||||
- If source is 慈悲 → "compassion" (correct)
|
||||
- If source is 悬壶济世 → "compassionate mission" (correct — the healing spirit)
|
||||
|
||||
### 生存层面 → making a living (WRONG)
|
||||
|
||||
生存层面 = the existential/survival dimension. Not just earning wages.
|
||||
→ "survival-level needs" or "the level of basic existence"
|
||||
|
||||
### 修学 → "practice" (drops study dimension)
|
||||
|
||||
修学 combines 修 (practice/cultivation) and 学 (study/learning). Rendering only
|
||||
as "practice" loses the study dimension. Use "practice and study" or
|
||||
"cultivation and learning" — especially in 静心学堂/Dharma study contexts where
|
||||
the academic dimension is emphasized.
|
||||
|
||||
### 信仰者 → "believers" (Christian connotation)
|
||||
|
||||
信仰者 = "people of faith" but in Buddhist context, "believers" carries Christian
|
||||
overtones. Use "practitioners" or "adherents" to avoid the implication. Reserve
|
||||
"believers" for contexts where the source explicitly uses 信徒 or where the
|
||||
Christian parallel is the point.
|
||||
|
||||
## Loss of Dharma meaning
|
||||
|
||||
### 生生增上 → continuously elevate our life (INCOMPLETE)
|
||||
|
||||
生生 = life after life (multi-life Buddhist perspective). The single-life rendering
|
||||
"continuously elevate our life" loses the Dharma meaning entirely.
|
||||
→ "continuously elevate our life, life after life"
|
||||
|
||||
## Degree / register shifts
|
||||
|
||||
### "tantamount to" drops 几乎
|
||||
|
||||
几乎 = "almost." "tantamount to" = "is in fact." Dropping 几乎 strengthens the
|
||||
claim. Source "几乎等同于一次全球化运动" = "almost equivalent to a globalization
|
||||
movement," NOT "tantamount to a globalization movement."
|
||||
|
||||
### "untenable" for 都是不行的 (overstates)
|
||||
|
||||
都是不行的 = "is not acceptable / won't do." "Untenable" = "indefensible" or
|
||||
"cannot be maintained." Too strong. Use "impermissible" or "is not acceptable."
|
||||
|
||||
### "monumental event" for 大事 (slight stretch)
|
||||
|
||||
大事 = "a major event" or "an event of importance." "Monumental" is typically
|
||||
reserved for tasks, errors, or achievements (e.g. "monumental task,"
|
||||
"monumental mistake"). "Event of great importance" or "landmark event" is more
|
||||
idiomatic.
|
||||
|
||||
### "perennial" for 永恒 (wrong register)
|
||||
|
||||
永恒 = eternal/ultimate. "Perennial" = recurring (per year, per season). They
|
||||
are not synonyms. Use "eternal" or "ultimate."
|
||||
|
||||
### 学部委员 → Member (UNDERSTATES)
|
||||
|
||||
学部委员 is CASS's highest academic title, equivalent to "Academician."
|
||||
"Member" understates the prestige significantly. → "Academician" or
|
||||
"Member of the Academic Divisions."
|
||||
|
||||
### 文明 → culture (WRONG)
|
||||
|
||||
文明 is "civilization," not 文化 "culture." When a text discusses 文明传播
|
||||
(civilizational transmission), do not substitute "cultural transmission."
|
||||
|
||||
### 教制建设 → reforming (ADDS CONNOTATION)
|
||||
|
||||
教制建设 means "developing/building monastic institutions." Adding "reforming"
|
||||
introduces a connotation of fixing something broken that is not in the source.
|
||||
→ "developing monastic institutions" or "institutional development."
|
||||
|
||||
### 一荣俱荣、一损俱损 → too loose
|
||||
|
||||
This idiom has a conditional structure: "if one prospers, all prosper; if one
|
||||
suffers, all suffer." Rendering as "thrived together and suffered together"
|
||||
loses the mutual-dependence logic. → "shared prosperity and adversity alike"
|
||||
or "rose and fell together."
|
||||
|
||||
### 成圣成贤 collapsing
|
||||
|
||||
圣 (sage) and 贤 (worthy) are distinct Confucian categories. Collapsing both
|
||||
to "a sage" loses the distinction. → "sagehood and worthiness" or "a sage or worthy."
|
||||
|
||||
### "must" overuse from 应当/倡导
|
||||
|
||||
Source patterns 应当 (should), 倡导 (advocate/champion), 今后要 (going forward, should)
|
||||
often get rendered as "must" by reflex. "must" in English is a strong directive
|
||||
appropriate only for 一定要, 必须, 务必. Default to "should" or "ought to" for
|
||||
recommendations. A whole section may be 倡导 without 一定 anywhere — don't
|
||||
manufacture urgency the source doesn't have.
|
||||
|
||||
### 文明 → culture (WRONG)
|
||||
|
||||
文明 is "civilization," not 文化 "culture." When a text discusses 文明传播
|
||||
(civilizational transmission), do not substitute "cultural transmission."
|
||||
|
||||
#### 因病返贫 → "back into poverty"
|
||||
"返贫" means becoming poor due to illness, not returning to previous poverty. Use "into poverty" or "driven into poverty."
|
||||
|
||||
#### Diacritics: use DB form, not academic Sanskrit
|
||||
| Wrong | Right | Source |
|
||||
|---|---|---|
|
||||
| `Mahāsthāmaprāpta` | `Mahasthamaprapta` | 佛教术语 |
|
||||
| `Yogācārabhūmi Śāstra` | `Yogacarabhumi-Sastra` | 经论名 |
|
||||
| `Avalokiteśvara` | `Guanyin` | 佛教术语 |
|
||||
| `pravāraṇā` | `Pavarana` | BAICKZ |
|
||||
|
||||
Exception: `Kṣitigarbha` — DoT定稿 uses diacritics, so keep them.
|
||||
When in doubt, search the DB and follow the highest-priority source. See dharma-translation skill `references/diacritics-convention.md`.
|
||||
|
||||
返贫 = become poor (from a non-poor state) due to medical costs. "Back" implies
|
||||
the person was previously poor — not necessarily true. This is about medical bankruptcy.
|
||||
→ "into poverty" or "fall into poverty" (no "back")
|
||||
|
||||
## Subject-shift calques
|
||||
|
||||
When the source uses an abstract/system noun as the grammatical subject (佛教,
|
||||
文明, 宗教, 文化) and the target reflexively substitutes a more concrete
|
||||
agent (Buddhist practitioners, civilization-builders, religious people, etc.),
|
||||
the English is wrong: the source is *not* talking about people, it's talking
|
||||
about the system.
|
||||
|
||||
**Pattern**: 佛教都被社会大众赋予期望 → EN drifts to "Buddhist practitioners
|
||||
are looked upon with hope." Wrong subject — source is 佛教, not 佛教徒. Right:
|
||||
"Buddhism is regarded with hope by the broader society."
|
||||
|
||||
**Detection**: For each translated sentence, find the *grammatical subject*
|
||||
in the English and check it matches the *grammatical subject* in the Chinese.
|
||||
If the English subject is a concrete agent and the Chinese is an abstract
|
||||
system noun, it's a subject-shift calque.
|
||||
|
||||
## Factual inconsistencies across paired descriptions
|
||||
|
||||
When the same person, place, or thing is described in two different paragraphs
|
||||
(intro + later reference), check that *every* descriptor matches: title,
|
||||
affiliation, role, credentials. A translation can be factually inconsistent
|
||||
even when each individual sentence is correct in isolation.
|
||||
|
||||
**Pattern**: Prof. Wei's credentials in paragraph 1: "Academician of the
|
||||
Chinese Academy of Social Sciences (CASS) and research fellow at the
|
||||
Institute of World Religions." Paragraph 5 (same person, same source
|
||||
reference): "a member of the CASS academic committee and Director of the CASS
|
||||
Buddhist Research Center." Two different titles for the same CASS affiliation.
|
||||
One is right, the other is wrong.
|
||||
|
||||
**Detection**: For each named person, build a dict {name → {credential:
|
||||
sentence_refs}} and check that all credentials in the dict match. Same
|
||||
institution or title can have different renderings in different paragraphs.
|
||||
|
||||
## Calque checklist (subtle English calques of Chinese verbs)
|
||||
|
||||
These are common Chinese-verb → English-verb pairs where the English word
|
||||
sounds natural in isolation but is a direct calque of the Chinese. They're
|
||||
easy to miss in a first-pass review because each one parses correctly:
|
||||
|
||||
| Chinese verb | Wrong (calque) | Right (idiomatic) |
|
||||
|---|---|---|
|
||||
| 赋予 (entrust with) | "look to with hope" | "regard with hope" |
|
||||
| 得到 (obtain) | "draw forth" | "draw on" / "gain" |
|
||||
| 发挥 (bring into play) | "bring into full play" | "make the most of" |
|
||||
| 承担 (assume) | "shoulder" | "take on" |
|
||||
| 重视 | "attach importance to" | "value" / "emphasize" |
|
||||
| 体现 | "embody" / "reflect" | "show" / "demonstrate" (when abstract) |
|
||||
|
||||
**Detection**: When the English uses an unusual verb that maps 1:1 to a
|
||||
Chinese word, and the Chinese word is a high-frequency academic verb (发挥,
|
||||
承担, 体现, 重视), check whether the English reads as a calque. A common
|
||||
smell: the English verb is "correct" but more formal/dramatic than the
|
||||
surrounding prose.
|
||||
|
||||
## Tonal coherence inside a parallel list
|
||||
|
||||
When a list of items in a section should have parallel structure (e.g. three
|
||||
"champion X, oppose Y" items; six "developing X, strengthening Y" items),
|
||||
check that the *verb choice* is consistent across the list. Inconsistency
|
||||
within a parallel structure is a strong signal of drift.
|
||||
|
||||
**Pattern**: Section V lists "First, Buddhism should serve as... Second, it
|
||||
must serve as... Third, it must serve as..." Source has 应当/要做 for all
|
||||
three. The English should match: all three "should serve as" or all three
|
||||
"must serve as." Mixing is a tell.
|
||||
|
||||
**Detection**: For each parallel-list section (First/Second/Third, etc.),
|
||||
extract the verb (or other repeated slot) and verify it's identical. Drift
|
||||
inside a parallel list is one of the easiest flow issues to catch
|
||||
mechanically — just look for variance.
|
||||
|
||||
## Multi-pass review structure
|
||||
|
||||
Translation review benefits from three distinct passes, run separately, each
|
||||
catching a different category of error:
|
||||
|
||||
1. **Pass 1: terminology + consistency + line count** — fast, mechanical.
|
||||
Catches: 人生佛教/人间佛教 conflation, 修行/修学, 恨→resentment,
|
||||
paired inconsistencies (Buddhism vs Buddhist practitioners), missing
|
||||
content, wrong numerals.
|
||||
|
||||
2. **Pass 2: mechanical/formatting** — em-dash convention, double punctuation,
|
||||
numbering mismatches, capitalisation typos, garbled text. Often skipped
|
||||
if the file "looks clean." This pass is what makes the file safe to
|
||||
publish; do it even when no content issues are obvious.
|
||||
|
||||
3. **Pass 3: flow/tonal/calques** — *read the full English as a piece of
|
||||
prose*. Catch: dramatic verbs that read as calques ("draw forth,"
|
||||
"into full play," "shoulder"), intensifier drift ("profoundly important"
|
||||
× 3 in one section), consistency of "must"/"should" inside parallel
|
||||
lists, subject misattribution in calque. Often the user will prompt
|
||||
this pass with "are you sure it reads well?" or "do the words hang
|
||||
together?" Treat that prompt as a signal to re-read the whole English
|
||||
target, not just spot-check.
|
||||
|
||||
Pass 3 in particular is the one that catches the *most embarrassing* errors
|
||||
— the ones where the English is grammatical and faithful but reads as
|
||||
"translationese." Don't skip it.
|
||||
|
||||
## DoT定稿 term drift
|
||||
|
||||
### 念死 → recollection of death (WRONG per DoT定稿)
|
||||
|
||||
DoT定稿 has "Cultivating mindfulness of death" / 佛教术语 has "contemplating the
|
||||
impermanence of death". The established term is "mindfulness of death", not
|
||||
"recollection of death." → "mindfulness of death" / "death-mindfulness"
|
||||
|
||||
### 三级修学 → Three-Level Study Program (WRONG per DoT定稿)
|
||||
|
||||
DoT定稿 has "Three-Stage Practice." → "Three-Stage Practice"
|
||||
|
||||
### 下士道/中士道/上士道
|
||||
|
||||
DoT定稿: "Path for Persons of Small/Medium/Great Capacity" — not "path of the
|
||||
initial/middle/great scope."
|
||||
|
||||
### 观音菩萨 → Avalokiteśvara (AVOID in MPI translations)
|
||||
|
||||
佛教术语 has "Guanshiyin/Guanyin Bodhisattva." Use "Guanyin Bodhisattva."
|
||||
|
||||
## Workflow pitfall
|
||||
|
||||
### Translating before consulting terms DB
|
||||
|
||||
|
||||
## Patterns from 人生百问 review (self-review findings)
|
||||
|
||||
The review of *人生百问* surfaced a set of recurring calques, register
|
||||
mismatches, and Buddhist-term choices that keep reappearing in MPI
|
||||
CN→EN work. Treat these as a supplemental checklist after the main
|
||||
translation-pitfalls list.
|
||||
|
||||
### Stiff / formal calques to soften
|
||||
|
||||
| Source pattern | Stiff / literal rendering | Smoother options |
|
||||
|---|---|---|
|
||||
| 热衷于 / 很热衷 | "keen on" | "eager to," "enthusiastically pursuing" |
|
||||
| 善用理性 | "use reason skillfully" | "use reason well," "use reason wisely" |
|
||||
| 随缘 | "responding to conditions" | "in harmony with conditions" (when it flows better) |
|
||||
| 人云亦云 | "serve as reference" | "provide guidance," "serve as a reference" |
|
||||
| 引起共鸣 | "arouse resonance" | "resonate," "strike a chord" |
|
||||
| 强势而偏执 | "forceful and opinionated" | "forceful and dogmatic," "domineering and inflexible" |
|
||||
| 执着外境 | "cling to all sorts of external pursuits" | "cling to all sorts of external conditions" |
|
||||
| 一时兴起 | "spur-of-the-moment impulse" | "spur-of-the-moment idea" |
|
||||
| 大龄青年 | "older unmarried young people" | "older singles," "unmarried adults" |
|
||||
| 更受欢迎 | "more welcome" | "more well-liked," "more welcomed by others" |
|
||||
| 想起来很急，做起来又不急 | "feel urgent but do not act urgently" | "urgent in thought but not in deed" |
|
||||
| 随大流 / 大众化的 | "popular" | "mass-oriented," "public," "for the general public" |
|
||||
| 养生之道 | "wisdom culture" | "wisdom tradition," "culture of wisdom" |
|
||||
| 以此作为标准 | "take X as the only standard" | "make X the only standard" (when the sentence is top-heavy) |
|
||||
|
||||
### 究竟 / 终极 / 更高层次
|
||||
|
||||
| Source | Awkward | Preferred |
|
||||
|---|---|---|
|
||||
| 更究竟 | "more ultimate" | "more complete," "more profound," "higher" |
|
||||
| 更究竟的成功 | "a more ultimate success" | "a more complete success," "a higher success" |
|
||||
|
||||
In conversational Q&A, 究竟 often reads better as "complete" or "profound"
|
||||
than as the technical "ultimate." Reserve "ultimate" for explicit doctrinal
|
||||
discussions of 究竟谛 / ultimate truth.
|
||||
|
||||
### Buddhist term register
|
||||
|
||||
| Source | Issue | Preferred guidance |
|
||||
|---|---|---|
|
||||
| 大自在 | "great freedom" loses the "ease" nuance | "great freedom and ease" |
|
||||
| 开启智慧 | "opened up boundless wisdom" is odd | "awakened," "brought forth," "unfolded" |
|
||||
| 迷惑 (as affliction) | "confusion" is too generic | "delusion" (in Buddhist contexts) |
|
||||
| 惑业 | "confusion and karma" | "delusion and karma" (check terms DB) |
|
||||
| 善知识 | "great wise teachers" is redundant | "great teachers," "wise teachers," "authentic teachers" |
|
||||
| 皈依三宝 | "take refuge hastily" | "take refuge lightly," "take refuge without being ready" |
|
||||
| 尽未来际 | "for all future time" is literal | "for all future lives," "throughout all future time" |
|
||||
| 患得患失 | "worried about personal gains and losses" | "anxious about gain and loss" |
|
||||
| religious community | vague | "Buddhist community," "Sangha" |
|
||||
| 报身 | "enjoyment body" vs "reward body" | Verify target convention; MPI often prefers "reward body" |
|
||||
| 化身 | "transformation body" is common | Verify against target convention |
|
||||
| 等流果 | "continuative result" | Standard: "result of equal outflow," "correlative effect" |
|
||||
| 四力 (忏悔) | "power of eradication / remedy" | Use standard four powers terminology: remorse, support, restraint, refuge/reliance |
|
||||
| 末法 | "Dharma-ending age" / "Age of Dharma Decline" | Check terms DB; avoid inventing new renderings |
|
||||
| 暇满 | "well-endowed human form" | Standard: "leisure and endowment," "precious human life" |
|
||||
| 定课 | "fixed chanting session" | "daily practice," "daily recitation" (context-dependent) |
|
||||
| 八步骤三种禅修 | "Eight Steps and Three Kinds of Meditation" | DoT定稿: "Eight Steps and Three Types of Meditation" |
|
||||
| 功德 | "merit and virtue" (overused) | "merit" or "virtue" depending on context |
|
||||
| 福田 | "field of blessing" | Standard: "field of merit/blessing" |
|
||||
| 止观 | "calm abiding and special insight" | Or "śamatha-vipaśyanā"; avoid inventing new compounds |
|
||||
| 皈依 / 发心 / 戒律 / 正见 | gerund lists | Noun forms in catalogues: "refuge, aspiration, precepts, right view" |
|
||||
| 凡夫心 | "ordinary mind" | "mind of an ordinary being" (avoid Chan "ordinary mind" ambiguity) |
|
||||
| 加持 | "empowers one another" | "supports and blesses one another" (retain religious nuance) |
|
||||
| 登地菩萨 | "attained the grounds" | Standard rendering |
|
||||
| 弟子相 | "marks of a disciple" | Standard rendering |
|
||||
| 无缘大慈、同体大悲 | — | "unconditional great compassion and the great sympathy of seeing others as oneself" |
|
||||
| 人成即佛成 | — | "When a Human Is Perfected, Buddhahood Is Perfected" |
|
||||
| 人生佛教 | — | "Buddhism for Human Life" (do not conflate with 人间佛教 / Humanistic Buddhism) |
|
||||
| 三级修学 | — | DoT定稿: "Three-Stage Practice" |
|
||||
| 同喜班 | — | "Tongxi Class" (MPI convention; verify if glossary requires translation) |
|
||||
|
||||
### Idioms and set phrases
|
||||
|
||||
| Source | Literal / awkward | Preferred |
|
||||
|---|---|---|
|
||||
| 妄念纷飞 / 妄想纷飞 | "delusive thoughts still fly about" / "random thoughts fly about" | "delusions run wild," "wandering thoughts arise" |
|
||||
| 望尘莫及 | "outdo animals by far" (wrong direction) | "animals cannot compare," "are left far behind" |
|
||||
| 脚踏两条船 / 骑墙 | "straddling two boats" | "riding two boats," "trying to walk two paths" |
|
||||
| 选择困难症 | "choice difficulty" | "decision paralysis," "choice paralysis" |
|
||||
| 树欲静而风不止 | "the tree wishing stillness while the wind keeps blowing" | "the tree may crave stillness, but the wind will not cease" |
|
||||
| 磨刀不误砍柴工 | — | "sharpen the ax before cutting wood" (standard) |
|
||||
| 泥菩萨过江 | "clay bodhisattva" | Allude to the idiom; may need explanatory note |
|
||||
| 鄙视链 | "contempt chains" | Loan-translation is acceptable; keep if context is clear |
|
||||
| 十年浩劫 | "decade of catastrophe" | International readers may need a gloss |
|
||||
| 红尘滚滚 | "rolling red dust" | Acceptable in Buddhist register; do not soften |
|
||||
| 因上努力，果上随缘 | — | "work hard on the causes and let the results unfold as they may" |
|
||||
| 朝不保夕 | "we may not last until evening" | "life is precarious" |
|
||||
| 本末倒置 | — | "put the cart before the horse" |
|
||||
| 润物细无声 | — | "quiet, pervasive influence, like rain soaking into the earth unnoticed" |
|
||||
| 有口无心 | — | "with their mouths but not their hearts" |
|
||||
| 人非圣贤，孰能无过 | "To err is human" | Acceptable, but note it loses the Buddhist register |
|
||||
| Standard Buddhist idiom | — | Always prefer established English Buddhist idiom when one exists (e.g. Diamond Sutra "lives" not "bodies") |
|
||||
|
||||
### Sentence structure / rhythm
|
||||
|
||||
| Pattern | Problem | Fix |
|
||||
|---|---|---|
|
||||
| 3+ clause sentences | Can't be read in one breath | Split at natural breaks, especially after "world" or "Dharma" |
|
||||
| Top-heavy sentences | Main clause arrives too late | Move the main clause forward |
|
||||
| "is it not too late to X" | Inverted clause is awkward | "it is not too late to X" |
|
||||
| "patterned, standardized" | "patterned" is odd in English | "structured, standardized" or "model-based, standardized" |
|
||||
| "way of management" | awkward collocation | "approach to management," "style of governance" |
|
||||
| "correction of X through correcting Y" | repetitive | "transform X by correcting Y" |
|
||||
| "special circumstances are encountered" | passive/abstract | "special circumstances may create an exception" |
|
||||
| "formed from the Buddha's wisdom and merit" | mechanical | "arises from," "embodied wisdom and merit" |
|
||||
| "with every sound entering the ears" | flat idiom | "with every syllable clearly heard" |
|
||||
| "influenced by the liberation of individuality" | wrong collocation | "emancipation of the individual," "individual liberation" |
|
||||
| "the optimization of life" | technical-sounding | "flourishing of life," "growth in life" |
|
||||
| "right-livelihood occupation" | redundant | "right livelihood" |
|
||||
| "when time is used more valuably" | awkward adverb | "when time is used more wisely" |
|
||||
| "doing great things" | OK | Keep plain; do not overtranslate 做大事 |
|
||||
| "speak merely to show off your own cleverness" | misses idiom | 逞口舌之快 → "speak for the thrill of it," "indulge in sharp talk" |
|
||||
| "the power of eradication... the power of remedy" | non-standard | Use standard four powers terminology |
|
||||
| "pincer attack from within and without" | too militaristic in pastoral context | "pressure from within and without," "assault from both sides" |
|
||||
| "stalemate" | OK but possibly cold | "stuck place," "impasse" in spiritual contexts |
|
||||
| "persevere in one sutra" | loses metaphor | "go deeply into one sutra" |
|
||||
| "muddle-headedly" | rare adverb | "in a muddled way," "without understanding" |
|
||||
| "thoroughly applied" | slightly off | "when proficiency develops" |
|
||||
| "Using relics to check seems a little too late" | OK but flat | "Using relics as a measure seems rather late in the day" |
|
||||
| "unknowable mysticism" | 玄学 ≠ mysticism | "unknowable mystery" |
|
||||
| "reliance relationship" | awkward | "teacher-student relationship" |
|
||||
| "every place becomes a place of practice" | repetitive | "everywhere becomes a field of practice" |
|
||||
| "keeping an ordinary mind when things go well and still keeping an ordinary mind when things go badly" | wordy | "keeping an ordinary mind in both good times and bad" |
|
||||
| "ordinary, lax, and indulgent mind" | acceptable but loose | "ordinary mind of laxity and indulgence" |
|
||||
|
||||
### Register: spoken Q&A vs. written Dharma
|
||||
|
||||
*人生百问* is conversational Q&A. Keep the English warm and direct:
|
||||
|
||||
| Stiff / written | Conversational |
|
||||
|---|---|
|
||||
| "I do not think so necessarily." | "Not necessarily." / "I don't think so." |
|
||||
| "bustle about for liberation" | "rush about," "busy themselves" |
|
||||
| "starting a family is for liberation from singlehood" | "liberation from being single" |
|
||||
| "Seen only from the surface" | "On the surface," "From a surface view" |
|
||||
| "If one is liberated, how can these dreams be realized?" | "...how can these dreams still be realized?" |
|
||||
| "I deeply know that..." | "I know full well that..." |
|
||||
| "For yourself, you must..." | "First, for yourself..." / "Personally, you must..." |
|
||||
| "I believe that the external conditions we attract will also change." | "the conditions we draw to ourselves will also change" |
|
||||
|
||||
### Parallelism and grammar fixes
|
||||
|
||||
| Issue | Example | Fix |
|
||||
|---|---|---|
|
||||
| Gerund/infinitive mismatch | "drawing near to wise teachers and to rely on them" | "draw near to wise teachers and rely on them" |
|
||||
| Missing "still" | "how can these dreams be realized?" | "how can these dreams still be realized?" |
|
||||
| Mixed singular/plural | "the largest debt, not all of them" | "the largest debt, not every debt" |
|
||||
| Redundant compound | "sharp and keen faculties" | "sharp faculties" or "keen faculties" |
|
||||
| "disorder" as verb | "'nao' is to disorder" | "'nao' is to disturb" |
|
||||
|
||||
### Meta-pattern: when the review produces `review-comments.dj`
|
||||
|
||||
After a self-review or other-review pass, scan the resulting
|
||||
`review-comments.dj` for **recurring patterns** and add them to this file
|
||||
and to `buddhist-terminology.md`. The review of *人生百问* began as a
|
||||
single article checklist and became a reusable pattern library; future
|
||||
reviews should do the same.
|
||||
|
||||
@@ -1,187 +0,0 @@
|
||||
---
|
||||
name: translation
|
||||
description: Translate Chinese↔English Buddhist/Dharma content — register guidance from Mindfulness Bell corpus, tone, voice, cultural bridging technique.
|
||||
inputs: source.dj (Chinese djot), or .docx via docx2dj.fish
|
||||
outputs: target.dj (English djot), bilingual.dj, edit-suggestions.dj
|
||||
---
|
||||
|
||||
# Translation
|
||||
|
||||
Core translation technique for Buddhist/Dharma content. Conventions (djot format,
|
||||
terms DB query, workflows, output format) are in AGENTS.md.
|
||||
|
||||
## Source context
|
||||
|
||||
Before translating, understand the source's format and delivery context. Is it a transcript of an oral talk, a book excerpt, a guided meditation script, a Q&A, a written article, or another genre? The register shapes the translation. If the context is not clear from the file path or source content, ask the user before proceeding.
|
||||
|
||||
## Mindfulness Bell Corpus
|
||||
|
||||
English Buddhist prose — register/style reference for translation.
|
||||
|
||||
- **PDFs + index**: `~/meta/www.files/public/The Mindfulness Bell/` — 6 issues (MB92–MB97, 2023–2026), `index.yaml` lists all articles by author/title/page
|
||||
- **Articles**: `~/documents/jingxin-lessons/Mindfulness Bell/MB{92..97}/*.md` — per-issue markdown files. Read with `read_file`.
|
||||
|
||||
Four registers, useful as style targets:
|
||||
|
||||
| Register | Example | Key features |
|
||||
|----------|---------|-------------|
|
||||
| Dharma talk | Thầy (MB94 "Roses and Garbage", MB97 "Go as a River") | Short sentences, concrete images, coined terms ("interbeing"), oral address, Sanskrit with narrative explanation |
|
||||
| Teaching lineage | Sister Đoan Nghiêm (MB93 "Our Patriarch Liễu Quán") | "We" voice, terms explained, cultural bridging ("like Jesus"), dates in narrative |
|
||||
| Personal narrative | Mick McEvoy (MB94 "Touching the True Nature") | First-person, confessional, borrowed Dharma vocabulary, vernacular |
|
||||
| Editorial | Brother Pháp Lưu (MB94 welcome letter) | Polished but warm, conceptual framing, "we" address |
|
||||
|
||||
Quick-find in index: Thầy talks → `"Thích Nhất Hạnh"` + page ≤ 10; teachings → `"Sister"` or `"Brother"`; narratives → first-page articles by non-monastics; lineage → `"Patriarch"` or `"ancestor"`.
|
||||
|
||||
## Translation Principles
|
||||
|
||||
1. **Terms**: Either keep Sanskrit w/ narrative explanation (bodhisattva, Māra) OR coin new English (interbeing, inter-are). Avoid clunky calques.
|
||||
2. **Cultural bridging**: Add bridges for Western readers. A Chinese text mentioning 孔子 can stay; explain the function. Đoan Nghiêm's "like Jesus" is the pattern.
|
||||
3. **Tone**: Chinese Dharma texts are typically more formal than English equivalents. Decide consciously: keep formality or warm up (Thầy style).
|
||||
4. **Voice**: Direct address ("you"), concrete images, and oral rhythm make Dharma land in English. Abstract noun chains (common in Chinese→English translationese) kill it.
|
||||
5. **Sutra quotes**: Use standard English Buddhist idiom. Check terse-idiom conventions (e.g., Diamond Sutra "lives" not "bodies").
|
||||
|
||||
## Pitfalls
|
||||
|
||||
### Pre-flight accuracy check
|
||||
|
||||
Before delivering a translation, run through the structured accuracy/readability
|
||||
taxonomy in `../self-review/references/common-issues-taxonomy.md`.
|
||||
Catching these before review saves iteration cycles:
|
||||
- Omission, over-literal renderings, over-free renderings, subject confusion,
|
||||
overtranslation, terminology errors
|
||||
- Long/nested sentences, passive voice clustering, nominalization, top-heavy
|
||||
structure, obscure word choices, weak transitions
|
||||
|
||||
### 1:1 line mapping
|
||||
|
||||
Each physical source line maps to exactly one physical target line. Never merge
|
||||
continuation lines (lines ending with ` ` soft breaks) into a single
|
||||
translation entry. The bilingual format preserves the original line structure —
|
||||
breaking this destroys alignment.
|
||||
|
||||
### Blank lines in bilingual
|
||||
|
||||
When creating an initial bilingual template from source only: blank source
|
||||
lines pass through as-is. Only non-blank lines get an empty target placeholder.
|
||||
Treating blank lines as content lines (adding target+separator) creates
|
||||
excessive blank clusters. See `references/bilingual-format.md`.
|
||||
|
||||
### Soft break markers
|
||||
|
||||
Trailing ` ` (two spaces) on source lines indicate soft line breaks (paragraph
|
||||
continuations). Preserve these markers on both source and target lines.
|
||||
|
||||
### Practice element lists: use noun forms
|
||||
|
||||
When translating lists of Buddhist practice elements — especially the five
|
||||
essentials (皈依、发心、戒律、正见、止观) or similar enumerated components —
|
||||
render each as a noun phrase, not a gerund. "Refuge, aspiration, precepts,
|
||||
right view, śamatha-vipaśyanā" — not "taking refuge, arousing aspiration."
|
||||
These are named components of a system, not actions being described. The same
|
||||
applies to any catalog-style listing (三学, 八正道 components, etc.).
|
||||
|
||||
### Terminology decisions: user is the source of truth
|
||||
|
||||
When the user proposes a non-standard rendering (e.g. "人间佛教 should be
|
||||
'Buddhism for daily lives'") or asks for your input ("although I don't know
|
||||
what X should be"), respond with a brief options table — strengths and
|
||||
weaknesses — and let them choose. Do NOT unilaterally commit to a rendering
|
||||
and start applying it across the file. The user often has a reason for their
|
||||
proposal (e.g. deliberate departure from terms DB convention) or a strong
|
||||
opinion they haven't voiced yet. Once they pick, then apply consistently and
|
||||
annotate in the inline `{% %}` comment WHY this rendering was chosen so future
|
||||
editors know it was deliberate, not a slip.
|
||||
|
||||
## Polishing (润色)
|
||||
|
||||
After translating, do a deliberate readability pass before submitting for review.
|
||||
Read the English aloud — if a sentence can't be spoken in one breath, fix it.
|
||||
|
||||
These are common patterns (not an exhaustive list). For the full taxonomy with
|
||||
more categories and examples, read
|
||||
`../self-review/references/common-issues-taxonomy.md`.
|
||||
|
||||
Examples of systematic adjustments:
|
||||
|
||||
1. **Passive → active**. Passive clusters (3+ per paragraph) are the top
|
||||
readability killer in CN→EN translation. `If meat consumption were reduced`
|
||||
→ `If people consume less meat`.
|
||||
2. **Nominalization → verb**. `placed emphasis on cultivating` → `emphasized
|
||||
cultivating`; `the application of` → `applying`.
|
||||
3. **Sentence splitting**. If a sentence has 3+ clauses and runs past one breath,
|
||||
split it. The source's period is not a contract — English readers need
|
||||
shorter breath units than Chinese readers.
|
||||
4. **Academic → plain**. `constitute` → `make up`; `facilitate` → `help`;
|
||||
`endeavor to` → `try to`; `in order to` → `to`. These texts are lectures
|
||||
and conversations, not journal articles.
|
||||
5. **Register match**. Dharma talks and dialogues should sound spoken —
|
||||
contractions, direct address, concrete images. If the English reads like a
|
||||
paper abstract, warm it up. Check against the MB corpus registers for the
|
||||
target genre.
|
||||
6. **Review-specific calques**. For *人生百问*-style Q&A, check the pattern
|
||||
tables in `../self-review/references/translation-pitfalls.md` for recurring
|
||||
stiff calques ("keen on," "more ultimate," "choice difficulty," etc.) and
|
||||
the Buddhist-term register notes in `../self-review/references/buddhist-terminology.md`.
|
||||
|
||||
Do NOT apply these mechanically — each is a judgment call. A passive may be
|
||||
correct when the agent is unknown; a nominalization may be the right technical
|
||||
term. The goal is natural English that matches the source's register, not a
|
||||
formulaic rewrite.
|
||||
|
||||
## Quality Gates (Before Declaring Done)
|
||||
|
||||
Run this self-check before you hand off a first-pass translation. The goal is to
|
||||
catch the most expensive errors while they are still cheap to fix. For the full
|
||||
post-translation review workflows, load `self-review` and `other-review`.
|
||||
|
||||
### Accuracy
|
||||
|
||||
- [ ] **Line parity**: source and target line counts match exactly.
|
||||
- [ ] **No missing content**: every Chinese paragraph, quote, poem, or rhetorical
|
||||
climax has a corresponding English rendering.
|
||||
- [ ] **No overtranslation**: no parenthetical expansions or explanations not in
|
||||
the source.
|
||||
- [ ] **Terminology**: key Buddhist terms checked against the MPI terms DB
|
||||
(`terms-search` skill). Consistent within the file.
|
||||
- [ ] **Source faithfulness**: no concepts added, no details dropped.
|
||||
|
||||
### Readability
|
||||
|
||||
- [ ] **Active voice**: abstract/subjectless passives converted to "we" or a
|
||||
concrete agent where possible.
|
||||
- [ ] **Noun → verb**: `the propagation of` → `spread`; `placed emphasis on` →
|
||||
`emphasized`.
|
||||
- [ ] **Sentence length**: no sentence that cannot be spoken in one breath; split
|
||||
at natural breaks.
|
||||
- [ ] **Plain vocabulary**: `constitute` → `is/make up`; `facilitate` → `help`;
|
||||
`endeavor to` → `try to`.
|
||||
- [ ] **Register**: match the genre — Dharma talks and dialogues should sound
|
||||
spoken, not like paper abstracts.
|
||||
- [ ] **Concrete over abstract**: `mode of existence` → `way of living`;
|
||||
`ideological content` → `ideas`.
|
||||
|
||||
### Final pass
|
||||
|
||||
Read the entire English target aloud. If anything stalls, rephrase it.
|
||||
|
||||
## Meditation / Mindfulness Content
|
||||
|
||||
When translating guided meditation scripts, exercise guides, or posture instructions
|
||||
(rather than Dharma talks), use a lighter workflow. See `references/meditation-translation.md`.
|
||||
|
||||
## Other Pitfalls
|
||||
|
||||
### article-specific scripts
|
||||
|
||||
`scripts/proofread-pdf.py` is hardcoded for 佛教徒的人生态度 — body-start
|
||||
markers, header patterns, slug regex. Do NOT reuse for other articles.
|
||||
Create article-specific scripts per `references/proofread-pdf-workflow.md`.
|
||||
|
||||
## References
|
||||
|
||||
- `references/meditation-translation.md` — lighter workflow for meditation/mindfulness content
|
||||
- `references/markdown-to-djot.md` — converting .docx.md to .dj for translation prep
|
||||
- `references/bilingual-format.md` — bilingual.dj layout: source/target adjacent, blank separator between pairs
|
||||
- `references/diacritics-convention.md` — diacritics rules
|
||||
- `references/proofread-pdf-workflow.md` — pattern for creating article-specific PDF-vs-DOCX comparison scripts
|
||||
- `../self-review/references/common-issues-taxonomy.md` (cross-skill) — structured accuracy/readability checklist for pre-flight review
|
||||
@@ -1,41 +0,0 @@
|
||||
# Bilingual DJ Format
|
||||
|
||||
## Layout
|
||||
|
||||
Each pair: source line immediately followed by target line. Blank line separates pairs.
|
||||
|
||||
```
|
||||
source-line
|
||||
target-line
|
||||
|
||||
source-line
|
||||
target-line
|
||||
```
|
||||
|
||||
NOT:
|
||||
```
|
||||
source-line
|
||||
← WRONG: extra blank between source and target
|
||||
target-line
|
||||
```
|
||||
|
||||
## Creating initial bilingual from source only
|
||||
|
||||
Only non-blank source lines get an empty target placeholder. Blank lines in the
|
||||
source pass through as-is and serve as natural pair separators.
|
||||
|
||||
```
|
||||
source-A
|
||||
|
||||
source-B
|
||||
```
|
||||
|
||||
The blank line between source-A and source-B is an original blank from the
|
||||
source — do NOT add an extra target+separator for it.
|
||||
|
||||
Pitfall: treating blank source lines as content lines creates 3+ consecutive
|
||||
blank lines (source-blank → target-blank → separator-blank). This is wrong.
|
||||
|
||||
## Verification
|
||||
|
||||
`non_blank_source_lines × 2 + total_source_lines = bilingual_line_count`
|
||||
@@ -1,13 +0,0 @@
|
||||
# Diacritics Convention
|
||||
|
||||
When in doubt, search the terms DB and use the highest-priority source's form.
|
||||
|
||||
| Rule | Examples |
|
||||
|---|---|
|
||||
| No diacritics (default) | `Mahasthamaprapta` (佛教术语), `Yogacarabhumi-Sastra` (经论名), `Guanyin` (佛教术语), `Pabongkhapa` (nti) |
|
||||
| With diacritics | `Kṣitigarbha` (DoT定稿 uses this form) |
|
||||
| Sanskrit terms | Keep standard romanization: `bodhicitta`, `bardo`, `Amitabha`, `prajñā` |
|
||||
|
||||
The DB uses simplified romanization. DoT定稿 is the authority — if it uses diacritics for a term, follow it. Otherwise strip them.
|
||||
|
||||
Pitfall: academic/pedantic diacritics (`Mahāsthāmaprāpta`, `Yogācārabhūmi Śāstra`, `Avalokiteśvara`) are common in general knowledge but wrong per MPI conventions.
|
||||
@@ -1,65 +0,0 @@
|
||||
# Markdown to Djot Conversion
|
||||
|
||||
When source material arrives as `.docx.md` (pandoc-converted from docx), convert to `.dj` for translation workflows.
|
||||
|
||||
## Splitting combined articles
|
||||
|
||||
If a single markdown file contains multiple articles (common when docx has two talks in one file), split at the article boundary before converting. Use `sed` by line number:
|
||||
|
||||
```bash
|
||||
sed -n '1,218p' combined.md > a1.md
|
||||
sed -n '220,282p' combined.md > a2.md
|
||||
```
|
||||
|
||||
## TOC stripping
|
||||
|
||||
Pandoc docx→md produces a markdown TOC with tab-separated page numbers:
|
||||
|
||||
```markdown
|
||||
[一、对佛教的感悟\t1](#一、对佛教的感悟)
|
||||
[二、佛教与人类文明\t5](#二、佛教与人类文明)
|
||||
```
|
||||
|
||||
Strip before conversion:
|
||||
|
||||
```bash
|
||||
sed -i '/^\[.*\t.*\](#.*)$/d' input.md
|
||||
```
|
||||
|
||||
Or in Python: skip lines matching `line.startswith("[") and "\t" in line and "](#" in line`.
|
||||
|
||||
## Heading anchor cleanup
|
||||
|
||||
Pandoc's docx→md conversion adds `{#heading-id}` anchors to every heading:
|
||||
|
||||
```markdown
|
||||
## 1．安宁疗护 {#1．安宁疗护}
|
||||
```
|
||||
|
||||
These must be stripped before markdown→djot conversion, otherwise pandoc's djot writer leaves stray `{#...}` lines in the output:
|
||||
|
||||
```bash
|
||||
sed 's/ {#[^}]*}//g' input.md > clean.md
|
||||
```
|
||||
|
||||
## Conversion command
|
||||
|
||||
```bash
|
||||
pandoc clean.md -f markdown -t djot --wrap=none -o output.dj
|
||||
```
|
||||
|
||||
`--wrap=none` prevents reflow of long paragraphs.
|
||||
|
||||
## Post-conversion cleanup
|
||||
|
||||
Pandoc may still leave stray `{#...}` lines in djot output. Remove them:
|
||||
|
||||
```bash
|
||||
sed -i '/^{#.*}$/d' output.dj
|
||||
```
|
||||
|
||||
## Pandoc artifacts
|
||||
|
||||
- Unicode `——` (U+2014 × 2) → `------` in djot (two em dashes, `---` each). This is correct djot syntax.
|
||||
- Markdown hard line breaks (trailing ` `) → `\\\n` in djot. Preserves original paragraph structure.
|
||||
- Pandoc normalizes heading IDs (strips `、` and other punctuation). Ignore; the stray-line cleanup handles it.
|
||||
@@ -1,43 +0,0 @@
|
||||
# Meditation / Mindfulness Content Translation
|
||||
|
||||
When the source is a guided meditation script, exercise guide, posture instruction,
|
||||
or breathing practice (rather than a Dharma talk, sutra commentary, or teaching text),
|
||||
use a lighter workflow than the full dharma-translation pipeline.
|
||||
|
||||
## Register
|
||||
|
||||
Default to warm, direct instructional voice (Thầy-adjacent):
|
||||
- Second-person address ("you")
|
||||
- Concrete images, sensory details
|
||||
- Oral rhythm, short sentences
|
||||
- Present tense, imperative mood
|
||||
|
||||
MB corpus consultation is NOT needed for register — this content type has its own
|
||||
well-established English conventions (yoga/meditation instructional voice).
|
||||
|
||||
## Terms
|
||||
|
||||
Terms DB lookup for Buddhist-mindfulness vocabulary is useful but limited to key terms:
|
||||
- 正念 → mindfulness
|
||||
- 觉知 → awareness
|
||||
- 无我 → depends on context: "non-self" for philosophical/Dharma content; "selflessly" for embodied/movement instruction where the sense is no separate controller imposing on the action
|
||||
- 中道 → Middle Way
|
||||
- 丹田 → dantian (keep as-is; well-known in meditation/qigong)
|
||||
|
||||
Context-sensitive terms:
|
||||
- 心 (xīn): in meditation/movement contexts it often means "mind/attention" not emotional "heart." 持心 means holding the mind with focused attention, not holding with emotion.
|
||||
- 念 (niàn): mindfulness/attention/recollection — context between these.
|
||||
- Buddhist philosophical terms (无我, 空, 缘起) in non-philosophical contexts (movement instruction, body scans) may need practical/concrete translations rather than doctrinal ones.
|
||||
|
||||
Skip deep terms alignment unless dense Dharma vocabulary (emptiness, dependent origination,
|
||||
Buddha-nature, etc.) appears in the text.
|
||||
|
||||
## Comparison files
|
||||
|
||||
Still create 对照.dj as usual. See comparison file format in this skill.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Don't add formatting the source doesn't have**: sub-section labels using `【】` in Chinese should become plain `[label]` in English, not `*[label]*` or `**[label]**`. Match the source's formatting level exactly.
|
||||
- **`**text**` is Markdown, not Djot**: Djot emphasis uses single asterisks (`*text*`). Never use double asterisks in `.dj` files.
|
||||
- **心 ≠ heart by default**: in meditation/movement contexts, 持心 = holding the mind with attention, not holding with emotion. Translate based on context, not dictionary defaults.
|
||||
@@ -1,50 +0,0 @@
|
||||
# PDF-vs-DOCX Proofread Workflow
|
||||
|
||||
Compare typeset PDF against the authoritative DOCX manuscript. Catch
|
||||
discrepancies introduced during typesetting: dropped words, terminology
|
||||
drift, repositioned phrases, extra content.
|
||||
|
||||
## Pattern
|
||||
|
||||
The existing `scripts/proofread-pdf.py` is article-specific (hardcoded to
|
||||
佛教徒的人生态度). For each new article, create a similarly-shaped script:
|
||||
|
||||
```
|
||||
translate-files/<article>/<name>-<hash>.py
|
||||
```
|
||||
|
||||
Hash = `md5('translate-files/<article>')[:6]`
|
||||
|
||||
## Script Structure
|
||||
|
||||
1. Convert DOCX to plain text: `pandoc docx -f docx -t plain --wrap=none`
|
||||
2. Convert PDF to plain text: `pdftotext -layout pdf`
|
||||
3. Extract body from DOCX — find body start marker (first sentence of body)
|
||||
4. Extract body from PDF — find same marker, filter out:
|
||||
- Page slugs: `文章名.*indd \d+`
|
||||
- Headers: `The Mindful Peace Academy Collection`, article title
|
||||
- Page numbers: `^\d{1,3}$`
|
||||
- Section numerals: `^(I|II|III|IV)$`
|
||||
- Section name lines: `^(Three Basic Elements|...|Conclusion)$`
|
||||
5. Join hyphenated line breaks (`word-` at end + lowercase continuation)
|
||||
6. Fix PDF artifacts: `L iving` → `Living`, `T\s+he` → `The`
|
||||
7. Normalize both (collapse whitespace, unify quotes/dashes)
|
||||
8. Compare with difflib.SequenceMatcher or sentence-level substring search
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- PDF hyphen joining drops the hyphen: `self-knowing` → `selfknowing`.
|
||||
This causes cascading word-level diff failures. Use sentence-level or
|
||||
chunk-based matching instead of word-by-word comparison.
|
||||
- Section headers (I, II, III) may be present in DOCX body but filtered
|
||||
from PDF — not real discrepancies.
|
||||
- The existing `scripts/proofread-pdf.py` is hardcoded for 佛教徒的人生态度.
|
||||
Do NOT reuse it for other articles without rewriting the body-start
|
||||
markers and filter patterns. Create article-specific scripts instead.
|
||||
- `git diff --word-diff` fails when one file is multi-line and the other
|
||||
is single-line. Normalize both to single-line first.
|
||||
|
||||
## Example
|
||||
|
||||
`translate-files/正念禅修十要素/ten-elements-c7fcd9.py` — extracts cleaned
|
||||
body from both sources, outputs to `/tmp/` for side-by-side diffing.
|
||||
@@ -1,16 +0,0 @@
|
||||
# CodeGraph data files
|
||||
# These are local to each machine and should not be committed
|
||||
|
||||
# Database
|
||||
*.db
|
||||
*.db-wal
|
||||
*.db-shm
|
||||
|
||||
# Cache
|
||||
cache/
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Hook markers
|
||||
.dirty
|
||||
@@ -1,4 +0,0 @@
|
||||
.git
|
||||
**/__pycache__
|
||||
**/__pypackages__
|
||||
fly.toml
|
||||
@@ -1,2 +0,0 @@
|
||||
__pycache__/
|
||||
__pypackages__/
|
||||
@@ -1,11 +0,0 @@
|
||||
FROM python:3.14-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install --no-cache-dir flask duckdb gunicorn
|
||||
|
||||
COPY . .
|
||||
|
||||
EXPOSE 8910
|
||||
|
||||
CMD ["gunicorn", "-b", "0.0.0.0:8910", "-w", "1", "--timeout", "30", "server:app"]
|
||||
@@ -1,2 +0,0 @@
|
||||
#!/usr/bin/fish
|
||||
fly deploy -y --depot=false
|
||||
@@ -1,20 +0,0 @@
|
||||
# fly.toml app configuration file generated for buddhist-terms-search on 2026-06-09T13:38:18+08:00
|
||||
#
|
||||
# See https://fly.io/docs/reference/configuration/ for information about how to use this file.
|
||||
#
|
||||
|
||||
app = 'buddhist-terms-search'
|
||||
primary_region = 'sin'
|
||||
|
||||
[http_service]
|
||||
internal_port = 8910
|
||||
force_https = true
|
||||
auto_stop_machines = 'stop'
|
||||
auto_start_machines = true
|
||||
min_machines_running = 0
|
||||
processes = ['app']
|
||||
|
||||
[[vm]]
|
||||
memory = '256mb'
|
||||
cpus = 1
|
||||
memory_mb = 256
|
||||
@@ -1,124 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Full-text search over MPI term database. Queries unified_terms_flat via DuckDB LIKE.
|
||||
|
||||
Module usage:
|
||||
from search import search_terms
|
||||
results = search_terms("空性")
|
||||
results = search_terms("空性", limit=5, loc="心经", src="佛教术语")
|
||||
# returns list of dicts: {zh, en, loc, source}
|
||||
|
||||
CLI usage:
|
||||
python search.py <query> [limit]
|
||||
python search.py 空性 loc:心经 src:公案
|
||||
"""
|
||||
|
||||
import sys
|
||||
import os
|
||||
import duckdb
|
||||
|
||||
DB = os.path.join(os.path.dirname(os.path.abspath(__file__)), "termlib.duckdb")
|
||||
|
||||
|
||||
def _connect():
|
||||
return duckdb.connect(DB, read_only=True)
|
||||
|
||||
|
||||
def _search_rows(con, query, loc=None, src=None, limit=None):
|
||||
terms = query.split() if query else []
|
||||
clauses = []
|
||||
params = []
|
||||
|
||||
for t in terms:
|
||||
like = f"%{t}%"
|
||||
clauses.append("(zh LIKE ? OR en LIKE ?)")
|
||||
params.extend([like, like])
|
||||
|
||||
where = " AND ".join(clauses) if clauses else "1=1"
|
||||
|
||||
if loc:
|
||||
where += " AND loc LIKE ?"
|
||||
params.append(f"%{loc}%")
|
||||
if src:
|
||||
where += " AND source = ?"
|
||||
params.append(src)
|
||||
|
||||
sql = f"SELECT zh, en, loc, source FROM unified_terms_flat WHERE {where}"
|
||||
if limit is not None:
|
||||
sql += " LIMIT ?"
|
||||
params.append(limit)
|
||||
|
||||
return con.execute(sql, params).fetchall()
|
||||
|
||||
|
||||
def search(query, loc=None, src=None, limit=20):
|
||||
"""Search the terms database. Returns list of {zh, en, loc, source} dicts.
|
||||
|
||||
query: str — space-separated search terms (AND logic)
|
||||
loc: str — filter by loc column (LIKE match)
|
||||
src: str — filter by source column (exact match)
|
||||
limit: int — max results (default 20)
|
||||
"""
|
||||
con = _connect()
|
||||
try:
|
||||
rows = _search_rows(con, query, loc=loc, src=src, limit=limit)
|
||||
return [
|
||||
{"zh": zh, "en": en, "loc": loc_val or "", "source": src_val}
|
||||
for zh, en, loc_val, src_val in rows
|
||||
]
|
||||
finally:
|
||||
con.close()
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) < 2:
|
||||
print("Usage: terms-search <query> [limit]")
|
||||
print(" queries: '空性', 'emptiness 中观'")
|
||||
print()
|
||||
print("Special prefixes (can be standalone or combined with query):")
|
||||
print(" loc:<source> — filter by loc (e.g. loc:心经)")
|
||||
print(" src:<table> — filter by source table (e.g. src:佛教术语)")
|
||||
print()
|
||||
print("Examples:")
|
||||
print(" terms-search 空性")
|
||||
print(" terms-search 'suffering 苦' loc:心经")
|
||||
print(" terms-search src:公案")
|
||||
sys.exit(1)
|
||||
|
||||
raw = sys.argv[1]
|
||||
limit = int(sys.argv[2]) if len(sys.argv) > 2 else 20
|
||||
|
||||
loc_filter = None
|
||||
src_filter = None
|
||||
query_parts = []
|
||||
|
||||
for token in raw.split():
|
||||
if token.startswith("loc:"):
|
||||
loc_filter = token[4:]
|
||||
elif token.startswith("src:"):
|
||||
src_filter = token[4:]
|
||||
else:
|
||||
query_parts.append(token)
|
||||
|
||||
query_str = " ".join(query_parts)
|
||||
|
||||
if not query_str and not loc_filter and not src_filter:
|
||||
print("No search terms or filters. Usage: terms-search <query> [limit]")
|
||||
return
|
||||
|
||||
results = search(query_str, loc=loc_filter, src=src_filter, limit=limit)
|
||||
|
||||
if not results:
|
||||
print(f"No results for: {raw}")
|
||||
return
|
||||
|
||||
print(f"Results: {len(results)}")
|
||||
print()
|
||||
for r in results:
|
||||
print(f"zh: {r['zh']}")
|
||||
print(f"en: {r['en']}")
|
||||
print(f"loc: {r['loc'] or '-'} | src: {r['source']}")
|
||||
print()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,73 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import sys, os
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
import duckdb
|
||||
from flask import Flask, request, jsonify, render_template
|
||||
import search as s
|
||||
|
||||
app = Flask(__name__)
|
||||
|
||||
def do_search(q, loc, src, limit):
|
||||
with duckdb.connect(s.DB, read_only=True) as con:
|
||||
rows = s.search(con, q, loc, src, limit)
|
||||
return [{'zh': r[0], 'en': r[1], 'loc': r[2] or None, 'source': r[3]} for r in rows]
|
||||
|
||||
def do_sources():
|
||||
with duckdb.connect(s.DB, read_only=True) as con:
|
||||
rows = con.execute(
|
||||
'SELECT source, COUNT(*) AS cnt FROM unified_terms_flat GROUP BY source ORDER BY cnt DESC'
|
||||
).fetchall()
|
||||
return [{'source': r[0], 'count': r[1]} for r in rows]
|
||||
|
||||
@app.route('/search')
|
||||
def search():
|
||||
try:
|
||||
q = request.args.get('q', '')
|
||||
loc = request.args.get('loc')
|
||||
src = request.args.get('src')
|
||||
limit = request.args.get('limit', type=int)
|
||||
results = do_search(q, loc, src, limit)
|
||||
return jsonify({'count': len(results), 'results': results})
|
||||
except Exception as e:
|
||||
return jsonify({'error': str(e)}), 500
|
||||
|
||||
@app.route('/sources')
|
||||
def sources():
|
||||
try:
|
||||
sources = do_sources()
|
||||
return jsonify(sources)
|
||||
except Exception as e:
|
||||
return jsonify({'error': str(e)}), 500
|
||||
|
||||
@app.route('/sources.html')
|
||||
def sources_html():
|
||||
try:
|
||||
sources = do_sources()
|
||||
total = sum(s['count'] for s in sources)
|
||||
return render_template('sources.html', sources=sources, total=total)
|
||||
except Exception as e:
|
||||
return render_template('sources.html', sources=[], total=0, error=str(e))
|
||||
|
||||
@app.route('/')
|
||||
def ui():
|
||||
q = request.args.get('q', '')
|
||||
loc = request.args.get('loc', '')
|
||||
src = request.args.get('src', '')
|
||||
limit = request.args.get('limit', '')
|
||||
searched = bool(request.args)
|
||||
results = []
|
||||
count = 0
|
||||
error = None
|
||||
if searched and (q or loc or src):
|
||||
try:
|
||||
lim = int(limit) if limit else None
|
||||
results = do_search(q, loc or None, src or None, lim)
|
||||
count = len(results)
|
||||
except Exception as e:
|
||||
error = str(e)
|
||||
return render_template('index.html', q=q, loc=loc, src=src, limit=limit,
|
||||
searched=searched, results=results, count=count, error=error)
|
||||
|
||||
if __name__ == '__main__':
|
||||
app.run(port=8910)
|
||||
@@ -1,30 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<meta charset="utf-8">
|
||||
<title>静心学堂中英文翻译对照表</title>
|
||||
<h1>静心学堂（非官方）中英文翻译对照表</h1>
|
||||
<p><a href="/">search</a> | <a href="/sources.html">sources</a></p>
|
||||
<form method="get">
|
||||
<input name="q" value="{{ q }}" placeholder="query (e.g. 空性 emptiness)">
|
||||
<input name="loc" value="{{ loc }}" placeholder="loc filter">
|
||||
<input name="src" value="{{ src }}" placeholder="src filter">
|
||||
<input name="limit" value="{{ limit }}" size="4" placeholder="limit">
|
||||
<button type="submit">Search</button>
|
||||
</form>
|
||||
|
||||
{% if searched %}
|
||||
{% if error %}<p>Error: {{ error }}</p>
|
||||
{% else %}
|
||||
<p>{{ count }} results</p>
|
||||
<table border="1" cellpadding="4" cellspacing="0">
|
||||
<tr><th>ZH</th><th>EN</th><th>LOC</th><th>SRC</th></tr>
|
||||
{% for r in results %}
|
||||
<tr>
|
||||
<td>{{ r.zh }}</td>
|
||||
<td>{{ r.en }}</td>
|
||||
<td>{{ r.loc or '' }}</td>
|
||||
<td>{{ r.source }}</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</table>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
@@ -1,19 +0,0 @@
|
||||
<!DOCTYPE html>
|
||||
<meta charset="utf-8">
|
||||
<title>Sources — 静心学堂翻译对照表</title>
|
||||
<h1>Sources</h1>
|
||||
<p><a href="/">search</a> | <a href="/sources.html">sources</a></p>
|
||||
|
||||
{% if error %}<p>Error: {{ error }}</p>
|
||||
{% else %}
|
||||
<table border="1" cellpadding="4" cellspacing="0">
|
||||
<tr><th>Source</th><th>Rows</th></tr>
|
||||
{% for s in sources %}
|
||||
<tr>
|
||||
<td><a href="/?src={{ s.source }}">{{ s.source }}</a></td>
|
||||
<td>{{ s.count }}</td>
|
||||
</tr>
|
||||
{% endfor %}
|
||||
<tr><th>Total</th><th>{{ total }}</th></tr>
|
||||
</table>
|
||||
{% endif %}
|
||||
@@ -0,0 +1,19 @@
|
||||
[x] Merge .omp/APPEND_SYSTEM.md into AGENTS.md
|
||||
[x] Use git submodule for toolkit/ with URL git@git.sr.ht:~iacore/mpi-translation-toolkit
|
||||
(push failed: remote may not exist or SSH key not configured; run `git push -u origin main` in toolkit/)
|
||||
[x] Rename terms-search/ -> terms-database/
|
||||
[x] Rename skill directories with mpi- prefix
|
||||
[x] Create toolkit/ and fold scripts/, references/, terms-database/, skills/ into it
|
||||
[x] Update all in-repo references
|
||||
[ ] Update external Hermes config:
|
||||
~/.hermes/config.yaml: skills.external_dirs -> $MPI_PROJECT_ROOT/toolkit/skills
|
||||
[x] Test
|
||||
- toolkit/terms-database/search.py runs
|
||||
- toolkit/scripts/gen-bilingual.py runs
|
||||
- no stale references to scripts/, references/, terms-search/, or skills/
|
||||
[x] Delete .omp/APPEND_SYSTEM.md
|
||||
|
||||
Next:
|
||||
- Replace the placeholder URL in .gitmodules with the real remote for toolkit/.
|
||||
- Push the toolkit/ submodule to its remote, then run `git submodule update --init`.
|
||||
- Update ~/.hermes/config.yaml as noted above.
|
||||
Submodule
+1
Submodule toolkit added at eaa5e0bf4b
@@ -1,7 +1,7 @@
|
||||
# Term Alignment Report
|
||||
|
||||
File: translated.dj
|
||||
Database: $MPI_PROJECT_ROOT/terms-search/termlib.duckdb
|
||||
Database: $MPI_PROJECT_ROOT/toolkit/terms-database/termlib.duckdb
|
||||
Date: 2026-06-09
|
||||
|
||||
## Summary
|
||||
|
||||
Reference in New Issue
Block a user