The C++ text input model used by Windows and Linux currently uses UTF-32. The intention was to facilitate handling of arrow keys, backspace/delete, etc., however since part of what is synchronized with the engine is cursor+selection offsets, and those offsets are defined in terms of UTF-16 code units, this causes very bad interactions with the framework-side model.
This converts to using UTF-16, rather than UTF-32, so that the offsets align with the framework. It also adds surrogate pair handling to the operations that adjust indexes, to avoid breaking surrogate pairs. (Arbitrary grapheme cluster handling is out of scope for this PR; while definitely desirable in the long term, surrogate pair handling is much more critical since improper handling yields invalid UTF-16, which breaks the text field).
This partially fixes https://github.com/flutter/flutter/issues/55014. A framework-side fix is also necessary (since currently both the engine and the framework attempt to handle arrow keys, which is another out-of-scope-for-this-PR issue), but even without the framework fix this dramatically improves the cursor behavior on Windows when there are surrogate pairs somewhere in the string since at least the two sides agree on what indexes mean.
Includes minor plumbing changes to the text input plumbing on Windows so that we're not pointlessly converting from UTF-16 to UTF-32 and then back to UTF-16.
Fixes a few issues with Windows text input:
- Filters out ASCII control characters
- Filters out lead surrogates, which aren't valid UTF-16 on their own so will cause assertion failures if sent to Flutter
- Adds a bandaid fix for a crash due to mismatches in indexing in the C++ and Dart text models. (A better fix would be to use UTF-16 and add surrogate pair handling to deletion and forward/back; that will be a later PR since it has a larger scope.)
Fixes https://github.com/flutter/flutter/issues/54879
Significantly improves the behavior of non-ASCII text input on Windows. Correctly
processes incoming character events as UTF-16, and for now uses UTF-32 for
the text model so that the existing index-based logic will work much more often.
Future work is still needed, but this will handle far more cases correctly.
Changes include:
- File structure
- Header guards
- Include paths
- Namespaces
- Integration with the engine's GN build
- Conversion from jsoncpp to rapidjson
- Style and clang-format adjustment to match engine repository
