In the current state, hyphenation in all languages than Sanskrit seems
to work (case-folding edge cases). Thus, we just disable Sanskrit.
Packed tries are implemented, but not the finite state machine
(space/speed tradeoff).
This commit contains a throw-away test app, which runs on the host.
I think I want to replace it with unit tests, but I'm including it in
the CL because it's useful during development.
Bug: 21562869
Bug: 21826930
Bug: 23317038
Bug: 23317904
Change-Id: I7479a565a4a062fa319651c2c14c0fa18c5ceaea
Previously, the standard C tolower() function was used, which
didn't support any characters beyond the basic ASCII letters.
Bug: 22506121
Change-Id: Ibb81121caa29be44fbb59aa98891e9faafc57592
This patch adds hyphenation using the Liang hyphenation algorithm,
similar to TeX. It also improves the optimized line breaker so that
it works correctly and efficiently even when the line width is not
constant (there is a specialization for constant width, which is
probably worthwhile, but performance TODOs remain).
Still to be done:
* hyphenator has many shortcuts, only tested with English
* interaction between punctuation and hyphenation is problematic
Change-Id: I2d94a1668ebc536398b7c43fcf486333eeb7c6aa
