Skip to content

feat(hyphenate): add trie based hyphenation package - #3494

Merged
diegomura merged 1 commit into
masterfrom
dm/hyphenate
Aug 16, 2026
Merged

feat(hyphenate): add trie based hyphenation package#3494
diegomura merged 1 commit into
masterfrom
dm/hyphenate

Conversation

@diegomura

Copy link
Copy Markdown
Owner

Adds @react-pdf/hyphenate, a Liang hyphenation engine with an entry point per language.

Patterns are compiled into a trie once and matched with a walk from each offset, instead of scanning every pattern in the set per word. Results are cached per language.

Pattern data is read from hyphen rather than vendored, so the language sets stay maintained upstream. The dependency is pinned to the 1.6 line because later releases ship pre-compiled tries in a different shape.

Usage

import { syllables, hyphenate } from '@react-pdf/hyphenate/en-us';

syllables('extraordinary'); // ['ex', 'tra', 'or', 'di', 'nary']
hyphenate('extraordinary'); // 'ex­tra­or­di­nary'

86 languages are generated at build time from the pattern sets, each its own entry point so bundlers only include what is imported.

Comparison

20,000 unique English words, each library in its own process, best of five. Sizes are an en-us bundle, minified and gzipped.

Library Matcher With en-us patterns Per word
@react-pdf/hyphenate 0.9 KB 19.4 KB 0.80 µs
hypher 0.2.5 1.9 KB 17.4 KB 1.08 µs
hyphenated 1.2.0 22.4 KB 3.36 µs
@lunarisapp/hyphen 1.2.1 all languages in one entry 3.46 µs
hyphen 1.6.4 2.0 KB 20.2 KB 70.77 µs

Notes

  • Output was checked against hyphen across 258,231 tokens: 0.115% differ, all words where the same pattern occurs twice close together (possesses becomes pos-sess-es rather than pos-sesses). hyphen skips those matches; this does not.
  • Pattern exceptions are not applied, matching hyphen 1.6's own behaviour — it seeds them under keys it never looks up.
  • Nothing consumes this yet. Wiring textkit to it is a follow-up.

Adds @react-pdf/hyphenate, a Liang hyphenation engine with an entry
point per language. Patterns are compiled into a trie once instead of
being scanned one by one, and results are cached per language.

Pattern data is read from hyphen rather than vendored, so the language
sets stay maintained upstream.
@changeset-bot

changeset-bot Bot commented Aug 16, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 01bc3a8

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@react-pdf/hyphenate Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@diegomura
diegomura merged commit 5e161ca into master Aug 16, 2026
14 checks passed
@diegomura
diegomura deleted the dm/hyphenate branch August 16, 2026 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant