68 Commits

Author SHA1 Message Date
Roozbeh Pournader
b830102259 Revert "Lookup glyph from color emoji font before and after ZWJ." am: 1aa0c40db9 am: de728a34c9
am: f83d57f7e5

Change-Id: Iebf945839891cc1ab561cf424ef917ac15e08842
2016-08-11 23:40:20 +00:00
Roozbeh Pournader
1aa0c40db9 Revert "Lookup glyph from color emoji font before and after ZWJ."
This reverts commit 9431bb96406121d2eb3fda8ca8209edc434f96a0.

Bug: 30815709
Change-Id: I057d9bcd05246e58894abb4e9633bd10f6fab211
2016-08-11 21:03:00 +00:00
Ben Wagner
e6adb66ae9 Move SkTypeface::CreateXXX to SkTypeface::MakeXXX.
Skia is moving to returning smart pointers from its factory methods.
This updates uses of SkTypeface::CreateXXX to SkTypeface::MakeXXX and
generally updates use of SkTypeface to sk_sp. This will allow for the
removal of the SK_SUPPORT_LEGACY_TYPEFACE_PTR define.

Change-Id: If3e600c6cb86080576667bc77d427da4f6560afa
2016-08-05 13:57:22 -04:00
Seigo Nonaka
9431bb9640 Lookup glyph from color emoji font before and after ZWJ.
Unicode recommends that the zwj sequences should be emoji presentation
even if they don't have the proper U+FE0F. Thus always lookup the glyph
for the code point before and after zwj as if they have a U+FE0F
variation selector.

Bug: 30056627
Change-Id: I03958a92337eaba4a8dd9c5be824b2665aa4a103
2016-07-13 14:12:06 -07:00
Seigo Nonaka
c2642f9a33 Treat U+2695, U+2640, U+2642 as emoji characters.
Bug: 29885295
Change-Id: I1bf191a46d05e7099265d863bae0523c50817d0b
2016-07-13 14:05:02 -07:00
Seigo Nonaka
fab99dde4b Add some gender balanced components in to the sticky whitelist.
FEMALE SIGN(U+2640), MALE SIGN(U+2642), StAFF OF AESCULAPIUS(U+2695)
will be used as the ZWJ sequenced in gender balanced emoji sequence.
To be in the same run with ZWJ, mark these emoji as sticky chracters.

With this fix, Female police officer sequence will be shown correctly
regardless of VS16.

Bug: 30026374
Change-Id: I503fc061eaa943d45208bb69e885151610c430ce
2016-07-13 14:01:41 -07:00
Seigo Nonaka
98487de547 Lookup glyph from color emoji font before and after ZWJ.
Unicode recommends that the zwj sequences should be emoji presentation
even if they don't have the proper U+FE0F. Thus always lookup the glyph
for the code point before and after zwj as if they have a U+FE0F
variation selector.

Bug: 30056627
Change-Id: I03958a92337eaba4a8dd9c5be824b2665aa4a103
2016-07-12 10:07:19 +09:00
Seigo Nonaka
1946f48e84 Treat U+2695, U+2640, U+2642 as emoji characters.
Bug: 29885295
Change-Id: I1bf191a46d05e7099265d863bae0523c50817d0b
2016-07-11 15:27:05 +09:00
Seigo Nonaka
435076ff2e Add some gender balanced components in to the sticky whitelist.
FEMALE SIGN(U+2640), MALE SIGN(U+2642), StAFF OF AESCULAPIUS(U+2695)
will be used as the ZWJ sequenced in gender balanced emoji sequence.
To be in the same run with ZWJ, mark these emoji as sticky chracters.

With this fix, Female police officer sequence will be shown correctly
regardless of VS16.

Bug: 30026374
Change-Id: I503fc061eaa943d45208bb69e885151610c430ce
2016-07-08 15:49:50 +09:00
Seigo Nonaka
524369b0c6 Fix build failure of minikin_perftest
stat.st_size is off_t not size_t, so need to cast to size_t before
compare it.

Change-Id: I6b742746fbb9f254701fc91e515c293f93f912c5
2016-06-28 09:37:17 +00:00
Seigo Nonaka
5e6bc85d69 Introduce FontCollection perftest
This CL introduces performance tests for FontCollection.

To support TTC file in /system/fonts, this CL also extends FontTestUtils

Bug:29142734
Change-Id: I9d8ad24ca55f61031b85623ab7c26234239e4f41
2016-06-28 08:52:35 +00:00
Seigo Nonaka
0ca4fb6d44 Add more native perf tests to minikin.
This CL introduces performance tests for following three modules:
- Hyphenator
- WordBreaker
- GraphemeBreak

During using UnicodeUtils, need to decouple it from gtest since
perftest doesn't have gtest dependencies.

Bug:29142734
Change-Id: I700c662fa7d0a52f19d8e93150ad1a85dc28769f
2016-06-28 16:55:42 +09:00
Seigo Nonaka
81c79d6e1e Fix test utilities
This fixes following three memory leaks in test utilities.
There is no problem in production code and this CL doesn't affect
any production behaviors.

- SkTypeface leaks due to forget calling SkSafeUnref in dtor.
- MinikinFontForTest leaks during constructing FontCollection.
- FontCollection leaks due to unnecessary AddRef.

Change-Id: I22e1e0307f1b2499296acb1aacc3ef66076a36e9
2016-06-21 16:47:25 +09:00
Seigo Nonaka
14e2d136aa Always use minikin namespace.
Here is a new policy of the namespace of minikin.
- All components should be in minikin namespace.
- All tests are also in minikin namespace and no anonymous namespace.

Bug: 29233740
Change-Id: I71a8a35049bb8d624f7a78797231e90fed1e2b8c
2016-06-11 00:11:17 +09:00
Seigo Nonaka
c970b3da26 Merge "Introduce initial minikin perftest" 2016-06-09 03:31:45 +00:00
Seigo Nonaka
c38b249ef7 Reconstruct the directory structure of minikin tests.
To add perftests and reuse some utility classes, reconstruct
test directory structure.

- Move unit tests from minikin/tests to minikin/tests/unittests
- Extract utilitiy classes to minikin/tests/utils which will be
  used by perftests eventually.

Change-Id: I5026b177934e72ae67d362ee888302037da2f808
2016-06-09 03:25:03 +00:00
Seigo Nonaka
08bfef04af Introduce initial minikin perftest
Bug: 28992567
Change-Id: Id14cc86761d6c5b408262fe8684e2b293a420a4f
2016-06-08 13:01:55 +09:00
Roozbeh Pournader
74b56175e5 Do not break after Myanmar viramas
This is to work around a bug in ICU's line breaker, which thinks
there is a valid line break between a Myanmar kinzi and a consonant.
See http://bugs.icu-project.org/trac/ticket/12561 for the ICU bug.

Bug: 28964845
Change-Id: I076ac15077e5627cbccf6732900bcc60d8596dda
2016-05-25 16:46:56 -07:00
Seigo Nonaka
77f4883453 Do not break before and after ZWJ.
The emoji list is generated from external/unicode/emoji-data.txt

Bug: 28248662
Change-Id: Ie49b3782505665d62c24371ca23d317ae5e9c5f7
2016-04-19 14:09:38 +00:00
Seigo Nonaka
994aa84f7b Returns hasVariationSelector true for VS15/VS16
Minikin has a special font fallback for VS15/VS16, so
hasVariationSelector for emojis with VS15/VS16 should always return
true.

Bug: 27531970
Change-Id: Ieebd58f48b135b6ec50d999df68dcc09b1284606
2016-04-15 12:45:04 +09:00
Seigo Nonaka
6c60831cfc Fix minikin_unittests
This CL fixes following test cases in minikin_tests
- FontFamilyTest.hasVariationSelectorTest
- HbFontCacheTest.getHbFontLockedTest
- HbFontCacheTest.purgeCacheTest

For the fix of FontFamilyTest.hasVariationSelectorTest, removing virtual
from GetUniqueId() in MinikinFont.  After [1], MinikinFont's destructor
started calling purgeHbCache() which calls virtual method,
MinikinFont::GetUniqueId().  Fortunately, the SkTypeface::uniqueID()
returns just internal value, so we can store it at the construction time
and use it instead of calling SkTypeface::uniqueID() every time.

This patch also changes purgeHbFont to purgeHbFontLocked, as all uses of
it were already under global mutex. This change avoids deadlock on
explicit unref, as when invoked by a Java finalizer from the Java object
that holds a reference to the font.

Some of the tests needed to change to using the ref counting protocol
rather than explicitly destructing font objects, as well.

[1] 9afcc6e2bd4d89e4e1deb6e18c3c4daca4e114fd

Bug: 28105730
Bug: 28105688
Change-Id: Ie5983c4869147dacabdca81af1605066cd680b3f
2016-04-11 13:33:35 -07:00
Raph Levien
aaa4e34702 Avoid copying of font table data
The hb_font_t object holds on to tables of font data, acquired through
the MinikinFont::GetTable interface, which is based on copying data
into caller-owned buffers. Now that we're caching lots of hb_font_t's,
the cost of these buffers is significant.

This patch moves to a different interface, inspired by HarfBuzz's
hb_reference_table API, where the font can provide a pointer to the
actual font data (which will often be mmap'ed, so it doesn't even
consume physical RAM).

Bug: 27860101
Change-Id: Id766ab16a8d342bf7322a90e076e801271d527d4
2016-04-08 08:26:38 -07:00
Seigo Nonaka
f3afe92def Support multiple locales for font language settings.
Some fonts support multiple scripts, for example, some fonts for
Korean supports not only "Kore" but also "Jamo".

To select fonts based on their multiple languages, this CL introduces
the following changes:
- Compares all languages of the font family and use the maximum score
  for font selection.
- Even if each language of the font family doesn't support the requested
  language, the font get score of 2 if the requested font is covered by
  all of the languages of the font family. For example, the font for
  "ko-Hang,ko-Hani" gets score of 2 for the requested language "ko-Kore".

Bug: 26687969

Change-Id: I7f13b51464c9b01982bb573251d77052b9ddbd70
2016-04-04 22:31:16 +09:00
Roozbeh Pournader
d8917c69a9 Do not allow line breaks before currency symbols
Implement the change proposed in UTC document L2/16-043R
(http://www.unicode.org/L2/L2016/16043r-line-break-pr-po.txt) to make
sure we do not break between letters and currency symbols.

Bug: 24959657
Change-Id: Ia29d0e5625f84870bd910d0c6e19036d17206704
2016-03-16 16:21:09 -07:00
Seigo Nonaka
cee83d4032 Suppress log span due to returning null for itemize result.
Bug: 26808815
Change-Id: I2a5a52f2c441d27c7ef270342b4ef93c3de9e56e
2016-03-03 15:51:55 -08:00
Seigo Nonaka
450e96c817 Break regional indicators at even numbered code points.
Bug: 23288449
Change-Id: If1419ff9e44e8e640616979bae88311f414b42a1
2016-03-02 17:44:14 -08:00
Raph Levien
1934c2c3cb Fix wrong conditions in isEmojiBase
I computed ranges using low <= c || c <= high, should be &&.

Bug: 26829153
Change-Id: Ic1002d90b6a408a0b415f2d117d0e57adcbc2fa9
2016-02-27 07:43:56 -08:00
Raph Levien
56840e8006 Suppress line breaks in emoji + modifier
An emoji base with an emoji modifier renders as a single glyph and
thus should not be a line break. Current (Unicode 8) logic does
indicate a line break, so we override the results of the ICU line
break iterator. The code references a proposal to improve Unicode
behavior; when that is adopted and we upgrade ICU accordingly, the
special-case code should be deleted, but the tests can remain.

Bug: 27343378
Change-Id: I5de9c53e9a34c503816f9131e3d894e6f7a57d13
2016-02-26 19:05:34 +00:00
Seigo Nonaka
0036da164e Use color font if skin tone is specified.
If skin tone is specified, the base emoji should be emoji style even
if it is text presentation default emoji.

This patch also removes wrong test case which expects default emoji
presentation but it is controlled by family order in /etc/fonts.xml
and there is no special logic for default presentation in minikin.
Thus the default presentation unit test should not be in minikin.

Bug: 27342346

Change-Id: I74a2b2feab4d559535049e368cfd833063cce81c
2016-02-26 11:26:30 +09:00
Raph Levien
94d5525508 Merge "Suppress grapheme cluster breaks in emoji with modifiers" into nyc-dev 2016-02-24 20:58:55 +00:00
Raph Levien
adfa580f1f Suppress grapheme cluster breaks in emoji with modifiers
An emoji with a modifier should be treated as a single grapheme, i.e.
it should not be possible to place the cursor between the base and
modifier.  This patch implements the proposed Rule GB9c from Mark
Davis's proposal entitled "Fixing breaking properties for emoji",
L2/16-011R3.

The patch also skips over variation sequences attached the to the
preceding character, for computing grapheme cluster boundaries.

Bug: 26829153
Change-Id: Iff5bc2bb8e5246223a017c7cf33acfbf63817f16
2016-02-24 12:43:36 -08:00
Seigo Nonaka
dbf5c2776b Merge "Support Hanb script." into nyc-dev 2016-02-23 03:57:58 +00:00
Raph Levien
d3f45892c7 Suppress linebreaks in emoji ZWJ sequences
Due to the way emoji ZWJ sequences are defined, the ICU line breaking
algorithm determines that there are valid line breaks inside the
sequence. This patch suppresses these line breaks.

This is an adaptation of I225ebebc0f4186e4b8f48fee399c4a62b3f0218a
into the nyc-dev branch.

Bug: 25433289
Change-Id: I84b50b1e6ef13d436965eab389659d02a30d100f
2016-02-18 15:00:24 -08:00
Seigo Nonaka
533a01ea84 Support Hanb script.
Hanb is a union of Han and Bopomofo.

Bug: 26687969
Change-Id: Ic696bcbbc9607f3842fd0115668b8e7bd917e62b
2016-02-18 18:28:52 +09:00
Seigo Nonaka
bcb024df56 Merge "Improve Paint.measureText and Paint.hasGlyph for variation sequences." into nyc-dev 2016-02-18 04:33:36 +00:00
Raph Levien
c88ef135fc Add penalty for breaks in URLs and email addresses
Recent changes have added special cases for line breaks within URLs
and email addresses. Such breaks are undesirable when they can be
avoided, but at other times are needed to avoid huge gaps, or indeed
to make the line fit at all.

This patch assigns a penalty for such breaks, equal to the hyphenation
penalty. The mechanism is currently very simple, but would be easy to
fine-tune based on more detailed information about break quality.

Bug: 20126487
Bug: 20566159
Change-Id: I0d3323897737a2850f1e734fa17b96b065eabd9c
2016-02-17 23:13:44 +00:00
Raph Levien
6d15657e4a Add line breaks to email addresses and URLs
This change adds accceptable line breaks according to sections 7.42
(Dividing URLs and e-mail addresses) and 14.12 (URLs or DOIs and line
breaks) of the Chicago Manual of Style (16th ed.). In general, these
place breaks before punctuation symbols, and suppresses them after
hyphens.

Bug: 20126487
Bug: 20566159
Change-Id: I2d07d516b920a506a2f718c38fb435c5eb1ee1f8
2016-02-17 23:12:48 +00:00
Raph Levien
9c4cc648ab Special-case URLs and email addresses for line breaking
Detect URLs and email addresses, and suppress both line breaking and
hyphenation within them.

Bug: 20126487
Bug: 20566159

Change-Id: I43629347a063dcf579e355e5b678d7195f453ad9
2016-02-17 23:11:46 +00:00
Seigo Nonaka
6b1c227da6 Improve Paint.measureText and Paint.hasGlyph for variation sequences.
Before this patch, the font fallback chain iterated all installed font
families if a variation selector was specified.

This CL narrows down the range of iteration.
To decide the font family for the variation sequence, we need to search
for both the variation sequence and its base code point.
The new range of the iteration is a union of them.

With this change, the running time of Paint.hasGlyph for the variation
sequence improves 50% and the running time of Paint.measureText for the
variation sequence improves 40% for the large text case on Nexus 6
userdebug.

Bug: 26784699
Bug: 11750374

Change-Id: Iced1349e3ca750821d8882c551551f65bb569794
2016-02-17 16:01:20 +09:00
Raph Levien
57b6dae989 Refine hyphenation around punctuation
Implement a WordBreaker that defines our concept of valid word
boundaries, customizing the ICU behavior. Currently, we suppress line
breaks at soft hyphens (these are handled specially). Also, the
new WordBreaker class has methods that determine the start and end
of the word (punctuation stripped) for the purpose of hyphenation.

This patch, in its current form, doesn't handle email addresses and
URLs specially, but the WordBreaker class is the correct place to do
so. Also, special case handling of hyphens and dashes is still done
in LineBreaker, but all of that should be moved to WordBreaker.

Bug: 20126487
Bug: 20566159
Change-Id: I492cbad963f9b74a2915f010dad46bb91f97b2fe
2016-02-16 22:05:07 -08:00
Seigo Nonaka
89e80237bc Improve Paint.hasGlyph performance by caching hb_font_t
It turned out that hb_font_t creation is not a lightweight operation.
Especially, Paint.hasGlyph creates hb_font_t for all existing fonts
every time. To improve the performance, cache hb_font_t instead
of hb_face_t.

Note that to calculate horizontal advance, MinikinPaint needs to be
associated with hb_font_t by calling hb_font_set_funcs. With this patch,
hb_font_set_funcs may be called multiple times for the same hb_font_t
object. However this is not an issue since MinikinPaint is unique
during layout.

Bug: 26784699

Change-Id: I516498ae9f0127d700fc9829327e9789845a1416
2016-02-04 12:26:10 +09:00
Seigo Nonaka
6f9966ea7c Introduce multiple language based font fallback.
The motivation of this CL is enhance the font fallback score design
to support multiple language font fallback.

This CL contains following changes:
- Break language based font score into two: script-based score and
  primary-language-based score.
- The primary-language-based score is 0 if the script-based score is 0.
  If the script-based score is not 0 and the primary language is the
  as same as the requested one, the font gets an extra score of 1.
- The language score gets a higher multiplier for languages higher in
  the locale list.

Bug: 25122318
Bug: 26168983
Change-Id: Ib999997a88e6977e341f4c325e2a1b41a59db2d5
2016-01-15 14:35:14 +09:00
Raph Levien
6638e05ac2 Tailor grapheme boundaries so sequence emoji are one grapheme
Make it so it's not possible to position the cursor inside an emoji
formed by a sequence including zero-width joiners.

Bug: 25368653
Change-Id: I67ec0874cd1505f3c82ab91492ffc3d39a52fae6
2016-01-13 12:52:38 -08:00
Seigo Nonaka
2342e5310f Fix lock assertion failures in unit test.
The assertion for the lock state has now activated by
I9c4b1e1f09c6793e387fbdb8bb654cc0a13c65d5.
This CL fixes the assertion failure in the unit tests by acquiring
lock before calling the functions.

Change-Id: I6a6afefb4de01e8610c2abfe6c779afa9442cc67
2016-01-06 22:49:33 +09:00
Seigo Nonaka
d337652233 Replace Qaae script with Zsye
The emoji variant script "Zsye" is registered in ISO 15924.

Bug: 26226285
Change-Id: Ibc2bc740d57c48f99b6f66b1ad7595bfa8c3cff4
2016-01-05 05:05:57 +00:00
Seigo Nonaka
96233efe4c Merge "Save all kind of script tags into FontLanguage." 2015-12-22 04:01:19 +00:00
Seigo Nonaka
5e995fb850 Save all kind of script tags into FontLanguage.
The main purpose of this CL is expanding FontLanguage to be able to
save full script tag. Previously, FontLangauge kept only limited script
tags. With this CL, FontLanguage keeps all script tags.

This CL contains the following changes:
- FontLanguage changes:
-- Moved to private directory not to be instantiated outside of Minikin.
-- Removed bool(), bits(), FontLanguage(uint32_t) methods which are no
   longer used.
-- Change the FontLanguage internal data structure.
-- Introduces script match logic.

- FontLanguages changes:
-- Moved to private directory not to be instantiated outside of Minikin.
-- This is now std::vector<FontLanguage>

- FontLanguageListCache changes:
-- Now FontLanguageListCache::getId through
   FontStyle::registerLanguageList is the only way to instantiate the
   FontLanguage.
-- Normalize input to be BCP47 compliant identifier by ICU.

Bug: 26168983
Change-Id: I8df992a6851021903478972601a9a5c9424b100c
2015-12-22 09:34:04 +09:00
Seigo Nonaka
c5ec243b3a No op build should not build minikin_tests
Previous CL[1] adds module names into LOCAL_ADDITIONAL_DEPENDENCIES, but
it was wrong. LOCAL_ADDITIONAL_DEPENDENCIES only accepts file path.

However, BUILD_PREBUILT doesn't provide a file path of the installed
font. So use custom tool and LOCAL_GENERATED_SOURCES instead.

Confirmed no-op build doesn't built minikin_tests and
continuous_native_tests.zip contains all necessary files.

[1]: I7d83abc077bce4e38fd93c7d607bc7e1f7871e6b

BUG: 26197092
Change-Id: I90e80036248ae72e0e0f9c6144a259f5f96ec9ce
2015-12-15 16:11:21 -08:00
Seigo Nonaka
43bb9743ca Remove script matching score from the font selection fallback.
Removing the extra score of 2 for the script matching from the font
fallback score calculation.  If the two langauges have different
scripts, we should treat them as different languages.

Change-Id: Ie0d6f27bd1086248895935a7bd01b5d404044ad0
2015-12-10 18:41:03 -08:00
Seigo Nonaka
efaf02ea3a Copy test font files into data directory.
To work native tests with additional font related files, copy
the files into /data/nativetest/minikin_tests/.
This copy only happens when the minikin_tests is built.
It is not an expected to copy the font files into the product image.

Change-Id: I7d83abc077bce4e38fd93c7d607bc7e1f7871e6b
2015-12-09 14:44:12 -08:00