iacore
7e8ba4b291
pptx-translate, chinese-text-normalize, dharma-translation: canonical location now ./skills/ with symlinks in ~/.hermes/skills/. translation-review: merged CSV/XLSX review + .dj comparison workflows into single SKILL.md. Added buddhist-terminology.md and terms-db-alignment.md references from Hermes version.
43 lines
2.5 KiB
Markdown
43 lines
2.5 KiB
Markdown
---
|
|
name: chinese-text-normalize
|
|
description: Normalize Chinese markdown files — remove extraneous mid-sentence line breaks from fixed-width exports while preserving TOC structures, section headers, and intentional paragraph breaks.
|
|
---
|
|
|
|
When Chinese text has hard line breaks at a fixed width (~20-25 chars) — common in PDF exports, OCR output, or poorly-converted documents — use this skill to join them into flowing paragraphs.
|
|
|
|
## Triggers
|
|
|
|
- User asks to "fix line breaks" or "remove extraneous breaks" in Chinese text
|
|
- Chinese markdown files with lines that break mid-sentence at a consistent short width
|
|
- Files with vertical TOC (single-char-per-line 【】 sections) that need preservation
|
|
|
|
## Approach
|
|
|
|
Run `scripts/normalize_breaks.py <directory>` — it processes all .md files in the directory.
|
|
|
|
The script handles three file patterns:
|
|
|
|
1. **Vertical TOC + fixed-width body** — Preserves the decorative single-char TOC section, joins body paragraphs, strips inline page numbers (standalone digits like "3", "4")
|
|
2. **Outline TOC with stray breaks** — Preserves numbered outline items (一、...、1、...、...... separators), joins body paragraphs
|
|
3. **Already in paragraph format** — No change (safe to run idempotently)
|
|
|
|
### What it preserves
|
|
|
|
- Vertical TOC: single CJK/punctuation lines with 【】 brackets
|
|
- Section headers: 【...】、## ...、# ...、一、二、三、...、1、2、3、...
|
|
- Outline TOC entries: short numbered lines, lines with ...... separators
|
|
- Blank lines as paragraph separators
|
|
|
|
### What it removes
|
|
|
|
- Mid-sentence hard line breaks (joins consecutive CJK body lines)
|
|
- Inline page numbers (standalone 1-2 digit lines)
|
|
- Trailing blank lines
|
|
|
|
## Pitfalls
|
|
|
|
- **TOC detection boundaries**: The vertical TOC end is detected by finding the first line with 3+ CJK characters. If a page number like "2" sits between TOC and body, it lands in the TOC section — harmless but visible.
|
|
- **Section headers without markers**: Plain-text section titles (e.g., "生命可以被设计的依据") without 【】 or number prefixes won't be detected as headers. They'll form standalone paragraphs separated by blank lines, which is fine as long as blank lines exist around them.
|
|
- **Wiki-link TOC files**: Files like a course index with [[wiki links]] are NOT prose and should be excluded. The script has no special handling — skip those files manually or restore from git.
|
|
- **Not for mixed CJK/English prose**: The script treats any line with CJK characters as body text. Mixed-language documents may need manual review.
|