The current text input manager is win32-specific due to its use of
IMM32. For UWP, we'll need a TSF implementation. Once that happens we'll
want to extract out a TextInputManager interface and add a separate UWP
implementation of this class.
This updates the Win32 desktop embedder to support input method (abbreviated IM
or IME) composing regions.
In contrast to languages such as English, where keyboard input is
managed keystroke-by-keystroke, languages such as Japanese require a
multi-step input process wherein the user begins a composing sequence,
during which point their keystrokes are captured by a system input
method and converted into a text sequence. During composing, the user is
able to edit the composing range and manage the conversion from keyboard
input to text before eventually committing the text to the underlying
text input field.
To illustrate this, in Japanese, this sequence might look something like
the following:
1. User types 'k'. The character 'k' is added to the composing region.
Typically, the text 'k' will be inserted inline into the underlying
text field but the composing range will be highlighted in some manner,
frequently with a highlight or underline.
2. User types 'a'. The composing range is replaced with the phonetic
kana character 'か' (ka). The composing range continues to be
highlighted.
3. User types 'k'. The character 'k' is appended to the composing
range such that the highlighted text is now 'かｋ'
4. User types 'u'. The trailing 'k' is replaced with the phonetic kana
character 'く' (ku) such that the composing range now reads 'かく'
The composing range continues to be highlighted.
5. The user presses the space bar to convert the kana characters to
kanji. The composing range is replaced with '書く' (kaku: to write).
6. The user presses the space bar again to show other conversions. The
user's configured input method (for example, ibus) pops up a
completions menu populated with alternatives such as 各 (kaku:
every), 描く (kaku: to draw), 核 (kaku: pit of a fruit, nucleus), 角
(kaku: angle), etc.
7. The user uses the arrow keys to navigate the completions menu and
select the alternative to input. As they do, the inline composing
region in the text field is updated. It continues to be highlighted
or underlined.
8. The user hits enter to commit the composing region. The text is
committed to the underlying text field and the visual highlighting is
removed.
9. If the user presses another key, a new composing sequence begins.
If a selection is present when composing begins, it is preserved until
the first keypress of input is received, at which point the selection is
deleted. If a composing sequence is aborted before the first keypress,
the selection is preserved. Creating a new selection (with the mouse,
for example) aborts composing and the composing region is automatically
committed. A composing range and selection, both with an extent, are
not permitted to co-exist.
During composing, keyboard navigation via the arrow keys, or home and
end (or equivalent shortcuts) is restricted to the composing range, as
are deletions via backspace and the delete key. This patch adds two new
private convenience methods, `editing_range` and `text_range`. The
former returns the range for which editing is currently active -- the
composing range, if composing, otherwise the full range of the text. The
latter, returns a range from position 0 (inclusive) to `text_.length()`
exclusive.
Windows IME support revolves around two main UI windows: the composition window
and the candidate window. The composition window is a system window overlaid
within the current window bounds which renders the composing string. Flutter
already renders this string itself, so we request that this window be hidden.
The candidate window is a system-rendered dropdown that displays all possible
conversions for the text in the composing region. Since the contents of this
window are specific to the particular IME in use, and because the user may have
installed one or more third-party IMEs, Flutter does not attempt to render this
as a widget itself, but rather delegates to the system-rendered window.
The lifecycle of IME composing begins follows the following event order:
1. WM_IME_SETCONTEXT: on window creation this event is received. We strip the
ISC_SHOWUICOMPOSITIONWINDOW bit from the event lparam before passing it to
DefWindowProc() in order to hide the composition window, which Flutter
already renders itself.
2. WM_IME_STARTCOMPOSITION: triggered whenever the user begins inputting new
text. We use this event to set Flutter's TextInputModel into composing mode.
3. WM_IME_COMPOSITION: triggered on each keypress as the user adds, replaces,
or deletes text in the composing region, navigates with their cursor within
the composing region, or selects a new conversion candidate from the
candidates list.
4. WM_IME_ENDCOMPOSITION: triggered when the user has finished editing the text
in the composing region and decides to commit or abort the composition.
Additionally, the following IME-related events are emitted but not yet handled:
* WM_INPUTLANGCHANGE: triggered whenever the user selects a new language using
the system language selection menu. Since there some language-specific
behaviours to IMEs, we may want to make use of this in the future.
* WM_IME_NOTIFY: triggered to notify of various status events such as opening
or closing the candidate window, setting the conversion mode, etc. None of
these are relevant to Flutter at the moment.
* WM_IME_REQUEST: triggered to notify of various commands/requests such as
triggering reconversion of text, which should begin composition mode, insert
the selected text into the composing region, and allow the user to select new
alternative candidates for the text in question before re-committing their
new selection. This patch doesn't support this feature, but it's an important
feature that we should support in future.
This changes the Windows text handling so that keyboard events are sent to the framework first for handling, and then passed to the text input plugin, so that the framework has a chance to handle keys before they get given to the text field.
This is complicated by the async nature of the interaction with the framework, since Windows wants a synchronous response. So, in this change, I always tell Windows that the event was handled, and if the framework (eventually) responds that it wasn't, then I synthesize a new event and send it with SendEvent.
I also added support for detecting "extended" keys, since that was missing, and converted the OnKey handlers in the API to return a bool to indicate whether or not they have handled the event.
During multi-step text input composing, such as with Chinese, Japanese,
and Korean text input, the framework sends embedders cursor rect updates
in the form of two messages:
* TextInput.setMarkedTextRect: notifies the embedder the size and
position of the composing text rect (or cursor when not composing) in
local coordinates.
* TextInput.setEditableSizeAndTransform: notifies the embedder of the
size of the EditableText and 4x4 transform matrix from local to
PipelineOwner.rootNode coordinates.
On receipt of either message, we cache a local copy on the
TextInputPlugin and notify the Win32FlutterWindow of the updated cursor
rect. In a followup patch, we update Win32FlutterWindow to implement the
Win32 input manager (IMM) calls required to position the IME candidates
window while editing.
Sends the flutter/settings update message to the engine after starting
it. For now values other than 24-hour time preference are hard-coded,
but dark mode support can be added later.
Fixes https://github.com/flutter/flutter/issues/65590
Switches the Windows embedding from the standard C API to the new proctable version, to allow for unit testing of the embedding layer separately from the embedder APIs implementation. This includes moving some engine messaging that was still in flutter_windows to the C++ engine class to better encapsulate the proc table.
The framework handles arrow keys, delete, and backspace (and with better
unicode support), so we shouldn't handle them at the embedding level.
Fixes#69202
The C++ wrapper's plugin registrar can own plugins to provided lifetime
management. However, plugins expect the registrar to be valid for the
life of the object, including during destruction, so any owned plugins
must be explicitly cleared before any registrar-specific destruction
happens.
Replaces selection_base() and selection_extent() with selection() and
SetSelection(int, int) with SetSelection(range).
This also adds the following convenience methods to TextRange:
* reversed()
* Contains(size_t position)
* Contains(const TextRange& range)
as well as operator== for use in unit tests. When Flutter migrates to
C++20, we can replace that method with a default declaration.
Previously, TextInputModel's SetEditingState method was a 1:1 mapping of
the underlying protocol used on the text input channel between the
framework and the engine. This breaks it up into two methods, which
allows the selection to be updated independently of the text, and avoids
tying the API the the underlying protocol.
This will become more important when we add additional state to support
composing regions for multi-step input methods such as those used for
Japanese.
SetText resets the selection rather than making a best-efforts attempt
to preserve it. This choice was primarily to keep the code simple and
make the API easier to reason about. An alternative would have been to
make a best-effort attempt to preserve the selection, potentially
clamping one or both to the end of the new string. In all cases where an
embedder resets the string, it is expected that they also have the
selection, so can call SetSelection with an updated selection if needed.
The Windows and GLFW embeddings both had cases of missing returns after
calling `Error` on a method channel result object, leading to incorrect
double-calls on the result if the error paths were ever hit.
Replaces the (temporary) compile-time option to pass engine switches
with the ability to pass them temporarily at runtime via environment
variables. This moves the recently-added code for doing this on Windows
to a shared location for use by all desktop embeddings.
This is enabled only for debug/profile to avoid potential issues with
tampering with released applications, but if there is a need for that in
the future it could be added (potentially with a whitelist, as is
currently used for Dart VM flags).
Temporarily adds a way to enable mirrors as a compile time option,
as is already provided in the Linux embedding, to provide a migration
path for the one remaining known need for compile-time options
that has been raised in flutter/flutter#38569.
Replaces the (temporary) compile-time option to pass engine switches
with the ability to pass them temporarily at runtime via environment
variables.
This is enabled only for debug/profile to avoid potential issues with
tampering with released applicaitons, but if there is a need for that in
the future it could be added (potentially with a whitelist, as is
currently used for Dart VM flags).
Windows portion of:
https://github.com/flutter/flutter/issues/38569https://github.com/flutter/flutter/issues/60393
Originally font change notification was handled by forwarding
WM_FONTCHANGE to the Flutter HWND, to avoid adding new API surface, but
that's not a good solution in a multi-window scenario, and it would
require a completely different solution for UWP. It also requires
non-obvious plumbing in the runner.
This replaces that with an explicit API, so that there's a clean and
obvious way for the runner to trigger this event.
Cleans up header order/grouping for consistency: associated header, C/C++ system/standard library headers, library headers, platform-specific #includes.
Adds <cstring> where strlen, memcpy are being used: there are a bunch of places we use them transitively.
Applies linter-required cleanups. Disables linter on one file due to included RapidJson header. See https://github.com/flutter/flutter/issues/65676
This patch does not cover flutter/shell/platform/darwin. There's a separate, slightly more intensive cleanup for those in progress.
We currently use a mix of C standard includes (e.g. limits.h) and their
C++ variants (e.g. climits). This migrates to a consistent style for all
cases where the C++ variants are acceptable, but leaves the C
equivalents in place where they are required, such as in the embedder
API and other headers that may be used from C.
Add copyright headers in a few files where they were missing.
Trim trailing blank comment line where present, for consistency with
other engine code.
Use the standard libtxt copyright header in one file where it differed
(extra (C) and comma compared to other files in libtxt).
This also amends tools/const_finder/test/const_finder_test.dart to look
for a const an additional four lines down to account for the copyright
header added to the test fixture.
- Standardize on lowercase for windows.h
- Don't define NOMINMAX before including windows.h in (some) public
wrapper headers, since it causes a warning when combined with setting
NOMINMAX at the build level, which is the more robust way to avoid
issues with min/max and windows.h
The Windows embedding was based on the GLFW embedding, which grew
organically from a singe-file implementation that used structs to manage
all of the important state. It is in the process of being converted to a
cleaner object-based architecture, but currently it is a hybrid of
objects and structs that have redundant data, making it very prone to
errors of forgetting to update pointers in multiple locations.
This reduces the remaining structs to only a single pointer to the
larger object that manages the responsibilities that handle is
associated with, so that there is no need to wire things together in
multiple places.
For now they continue to exist as projections of the larger objects, but
that will be eliminated over time by having an object structure that
better reflects the API structure.
Fixes https://github.com/flutter/flutter/issues/64250
The response APIs for method channels and event channels used pointers
for optional parameters; this kept the API surface simple, but meant
that they couldn't take rvalues. As a result, returning success values
or error details often took an extra line, declaring a variable for the
result just to have something to pass the address of.
This converts them to using references, with function overloading to
allow for optional parameters, so that values can be inlined.
For now the pointer versions are still present, so that conversion can
be done before it becomes a breaking change; they will be removed soon.
Part of https://github.com/flutter/flutter/issues/63975
Adds APIs for runners to delegate WindowProc handlers into the Flutter
engine, and for plugins to register as possible delegates.
This allows for plugins to alter top-level window behavior in ways that
can only be done from the WindowProc, such as resize control. This
functionality remains entirely on the native side, so is synchronous.
Part of https://github.com/flutter/flutter/issues/53168
Relands https://github.com/flutter/engine/pull/20399
Makes BinaryMessenger available from FlutterEngine, rather than just the plugin registrar. This allows for method channels directly in applications without building them as plugins, and matches the other platforms.
Requires some restructuring of code and GN targets in the client wrappers to make the internals in the shared section usable by the implementations of platform-specific parts of the wrappers. Also fixes a latent issue with EnableInputBlocking symbols being declared but not defined for Windows that came up during testing of the restructing.
Fixes https://github.com/flutter/flutter/issues/62871
Makes BinaryMessenger available from FlutterEngine, rather than just the plugin registrar. This allows for method channels directly in applications without building them as plugins, and matches the other platforms.
Requires some restructuring of code and GN targets in the client wrappers to make the internals in the shared section usable by the implementations of platform-specific parts of the wrappers. Also fixes a latent issue with EnableInputBlocking symbols being declared but not defined for Windows that came up during testing of the restructuring.
Fixes https://github.com/flutter/flutter/issues/62871
Changes the interaction between the view controller and engine in both the C API and
the engine API, so that there's always an engine (as on other platforms) rather than
the engine APIs being specific to headless mode.
While adjusting the C API, this does a large cleanup:
- Renames all methods to follow a `FlutterDesktop` (prefix) + "class" name + method-style name.
E.g., `FlutterDestkopViewControllerCreate` rather than `FlutterDesktopCreateViewController`.
This makes it easier to see what functions operate on which conceptual "object" in the API.
- Reorders and groups them by the object they operate on.
Fixes https://github.com/flutter/flutter/issues/61966
Refactors the Windows embedding internals to make an engine object that
owns things associated with the engine rather than the view, and updates
the API surface to allow using the engine directly.
This is an incremental step toward both a cleaner, non-struct-based
internal structure and a finalized API surface.
Instead of a hand-rolled discriminated union (originally used to avoid a C++17
dependency, which is no longer an issue), implement EncodableValue as a
std::variant. Rather than simply changing the internals, this makes EncodableValue
a minimal std::variant subclass with only a handful of added methodS, replacing
the old IsFoo/FooValue APIs with the standard std::holds_alternative/std::get,
so that plugin code will use a standard-based API rather than a Flutter-specific
API for wrapped values.
This is a breaking change for Windows and GLFW plugins. In the short
term USE_LEGACY_ENCODABLE_VALUE can be set in builds to use the old
version, to separate rolling from updating.
Fixes https://github.com/flutter/flutter/issues/61970
The Windows, Linux, and GLFW embeddings (which all share a common code
ancestry) pass TextInput.setEditingState selection base and extents
straight through to the shared text model class. The model expects those
values to be valid, but the framework sends -1/-1 for "invalid"
selections, which happen for some empty text cases (e.g.,
TextFieldController.clear()).
This translates those invalid selection values to an empty selection at
the start of the string, as expected by the model.
Fixes https://github.com/flutter/flutter/issues/59140
For error messages in the Windows embedding, use stderr rather than
Windows debug logging, so that it will go to the console where, e.g.,
'flutter run' will display it.
The embedder.h API layer is an implementation detail of the desktop
embeddings, not part of the public API surface, so should not be part of
the public symbol list for those libraries.
- Adds a way to provide an AOT library to the C API.
- Adds app.so to the information provided by DartProject.
- Fixes the engine to only do static snapshot linking for Windows in
debug mode, not all modes, so that the provided library is used.
Engine side of https://github.com/flutter/flutter/issues/38477
This does some long-overdue refactoring of the spaghetti code that grew in the GLFW embedding to begin providing a clearer separation between the engine and the window. It is now possible to register plugins, and run the runloop, on a headless engine, which makes headless mode much more usable. This is useful in some automated testing environments.
There is more refactoring that should be done in the future, but this is a good incremental point to stop as the PR is already large, and it provides useful new functionality as-is.
