Fixes an issue with Korean IMEs wherein a text input state update may be
sent to the framework that misleads the framework into assuming that IME
composing has ended.
When inputting Korean text, characters are built up keystroke by
keystroke until the point that either:
* the user presses space/enter to terminate composing and commit the
character, or;
* the user presses a key such that the character currently being
composed cannot be modified further, and the IME determines that the
user has begun composing the next character.
The following is an example sequence of events for the latter case:
1. User presses ㅂ. GCS_COMPSTR event received with ㅂ. Embedder sends
state update to framework.
2. User presses ㅏ. GCS_COMPSTR event received with 바. Embedder sends
state update to framework.
3. User presses ㄴ. GCS_COMPSTR event received with 반. Embedder sends
state update to framework.
4. User presses ㅏ. At this point, the current character being composed
(반) cannot be modified in a meaningful way, and the IME determines
that the user is typing 바 followed by 나. GCS_RESULTSTR event
received with 바, immediately followed by GCS_COMPSTR event with 나.
In step 4, we previously sent two events to the framework, one
immediately after the other:
* GCS_RESULTSTR triggers the text input model to commit the current
composing region to the string under edit. This causes the composing
region to collapse to an empty range.
* GCS_COMPSTR triggers the text input model to insert the new composing
character and set the composing region to that character.
Conceptually, this is an atomic operation. The fourth keystroke causes
the 반 character to be broken into two (바 and ㄴ) and the latter to be
modified to 나. From the user's point of view, as well as from the IME's
point of view, the user has NOT stopped composing, and the composing
region has simply moved on to the next character.
Flutter has no concept of whether the user is composing or not other
that whether a non-empty composing region exists. As such, sending a
state update after the GCS_RESULTSTR event misleads the framework into
believing that composing has ended. This triggers a serious bug:
Text fields with input formatters applied do not perform input
formatting updates while composing is active; instead they wait until
composing has ended to apply any formatting. The previous behaviour
would thus trigger input formatters to be applied each time the user
input caused a new character to be input. This has the add-on negative
effect that once formatting has been applied, it sends an update back to
the embedder so that the native OS text input state can be updated.
However, since the GCS_RESULTSTR event is _immediately_ followed by a
GCS_COMPSTR, the state has changed in the meantime, and the embedder is
left processing an update (the intermediate state sent after the
GCS_RESULTSTR) which is now out of date (i.e. missing the new state from
the GCS_COMPSTR event).
Since GCS_RESULTR events are always immediately followed by a subsequent
GCS_COMPSTR (in the case where composing continues) or a
WM_IME_ENDCOMPOSITION (in the case where composing is finished), and
because the event handlers for both of those send updated state to the
framework, this change eliminates sending the (intermediate) state in
response to GCS_COMPSTR events.
Issue: https://github.com/flutter/flutter/issues/96209 (full fix)
Issue: https://github.com/flutter/flutter/issues/88645 (partial fix)
IAccessible objects should implement accHitTest. Our implementation, in
AXPlatformNodeWin, delegates hit testing to
AXPlatformNodeDelegate::HitTestSync. Here, we do a quick recursive
depth-first walk of the accessibility tree to return the best match.
We define the best match as the deepest node in the tree that contains
the point under test whose children do not.
Issue: https://github.com/flutter/flutter/issues/77838
The default implementation of GetUniqueId on ui::AXPlatformNodeDelegate
always returns ID 1. We had previously implemented this on the windows
platform node delegate, but for consistency's sake, and because the
default implementation is surprising, we're promoting this to the
FlutterPlatformNodeDelegate base class.
Issue: https://github.com/flutter/flutter/issues/77838
In the Win32 accessibility tree, each AXTree node has an associated
IAccessible object. In WindowWin32, our WM_GETOBJECT handler returns the
IAccessible associated with the root node of the tree (node 0). On other
platforms, we often add our root accessibility object as a subnode of
some existing accessibility object associated with the view. On Windows,
the root IAccessible _is_ the accessibility object associated with the
view (on Windows, and HWND).
In the previous implementation, AccessibleObjectFromWindow actually just
returns the root IAccessible object, which is equivalent to just
returning GetNativeViewAccessible. Instead, we just return null once we
hit the root of the tree.
Issue: https://github.com/flutter/flutter/issues/77838
On receipt of a FOCUS_CHANGED event from the AX tree, call
IAccessible::accFocus to tell screen readers to move the accessibility
focus to that node. This is in addition to setting the keyboard focus
via the EVENT_OBJECT_FOCUS event.
Issue: https://github.com/flutter/flutter/issues/77838
Implements OnAccessibilityEvent, which translates events from the AXTree
into Windows MSAA event notifications.
This also eliminates the UIA root object lookup in response to
WM_GETOBJECT, since our initial implementation, like Chromium's default
implementation, is based on MSAA.
Issue: https://github.com/flutter/flutter/issues/77838
`AXPlatformNodeDelegate` provides a default implementation of
`GetUniqueId()` which returns the same value (1) for all nodes.
`AXPlatformNodeWin` relies on a unique ID being returned from this
method in its `GetTargetFromChildID` method.
In the world MSAA accessibility, each `HWND` may have a root
`IAccessible` object associated with it. Events are fired by calling the
`NotifyWinEvent` Win32 call and specifying:
1. The `HWND` (in our case, the one associated with our `WindowWin32`)
2. The target object (`OBJID_CLIENT` for the root `IAccessible`
associated with that `HWND`)
3. A child identifier that uniquely identifies the target node in the
tree
This child identifier can be one of:
* `CHILDID_SELF` for events targeting the root `IAccessible`.
* A positive integer that specifies the index of a direct child of the
root.
* A negative integer which, when negated, specifies the unique ID of a
child within the tree.
On receipt of an accessibility event, `AXPlatformWin` (our `IAccessible`
implementation) looks up the target `IAccessible` from the child ID
specified on the event using its `GetTargetFromChildID` method and
delegates the appropriate method call to that object.
So, why is this implemented on our `AXPlatformNodeDelegate` subclass and
not on `AXPlatformNodeWin` itself? Because the implementation of
`AXPlatformNode::GetUniqueId()` delegates this lookup to the node
delegate.
As background, `AXUniqueId` automatically assigns itself a
globally-unique value in its constructor; it tracks all used values in
an internal static pool.
Issue: https://github.com/flutter/flutter/issues/77838
Implements DispatchAccessibilityAction: a pass-through method that
forwards to the engine, which calls through the Embedder API back to the
framework.
Issue: https://github.com/flutter/flutter/issues/77838
In the Windows embedder, the data parameter of
FlutterWindowsEngine::DispatchSemanticsAction was a vector of bytes.
Here, we switch it to use MallocMapping since that is the type of the
data paramater on the DispatchAccessibilityAction method of
AccessibilityBridgeDelegate, which calls this method. This avoids an
extra type conversion.
Adds AddSemanticsNodeUpdateWithChildren, which tests that the native
tree of IAccessible COM objects is an accurate representation of the
platform-agnostic accessibility tree.
Issue: https://github.com/flutter/flutter/issues/77838
* Win32 a11y bridge and platform node delegates
This is the third in a series of patches adding accessibility support
for the Windows embedder. This patch wires in the Accessibility bridge,
and lands the core structure of the Windows FlutterPlatformNodeDelegate
and AccessibilityBridgeDelegate classes, including:
* Instantiating the AccessibilityBridge when the semantics tree is
enabled.
* Creating FlutterPlatformNodeDelegate wrappers for semantics tree
nodes.
* Handling custom action updates.
* Building and updating the accessibility tree on semantics updates.
* Returning the native IAccessible objects when queried.
Breaking this out so that the follow-up patches can be reviewed and
landed in smaller, independent chunks.
Issue: https://github.com/flutter/flutter/issues/77838
Issue: https://github.com/flutter/flutter/issues/93928
Previously, we'd mocked out UpdateSemanticsEnabled using a simple
lambda. Due to the embedder API's C ABI, we were unable to make use of
lambda captures. This patch instead uses MOCK_ENGINE_PROC so we can make
the test variable a local and rely on lambda capture.
Issue: https://github.com/flutter/flutter/issues/77838
It turns out that unlike the Microsoft IME, Sogou sends a single IME_COMPOSITION message that has both the GCS_COMPSTR and GCS_RESULTSTR flags set, and we check for a composition string before result string, with an "else" clause to check for the result string, so if GCS_COMPSTR is set, we never check for a result string.
Windows does not provide a system notification the embedder can
subscribe to to determine when a screen reader or other assistive
technology that requires the semantics tree has been enabled. [1]
In the absence of such a notification, we watch for queries for the
IAccessible COM object representing the root node of the semantics tree,
and on receipt of such queries we enable the semantics tree. For the
time being, we assume that if accessiblity is enabled, it will be
enabled for the duration of the application lifetime. In a future patch
we can optimise this by adopting the approach taken by Chromium, which
is to disable semantics if we haven't received a query within some
reasonable timeout.
This patch enables Flutter's semantics tree; a follow-up patch wires in
the AccessibilityBridge which maintains the AXTree of COM objects that
is the Windows representation of the Flutter SemanticsTree.
Issue: https://github.com/flutter/flutter/issues/77838
[1]: https://docs.microsoft.com/en-us/windows/win32/winauto/screen-reader-parameter
When a text input client is cleared, such as when the user changes focus
from one text input widget to another, or when the window itself loses
focus, cancel IME composing mode, close the composing candidates window
(if any), and reset the composing text.
Issue: https://github.com/flutter/flutter/issues/90503
* Web
* macos
* Linux
* Easier web impl
* doc and format
* Better linux impl
* Format
* Better impl mac
* Format
* Windows
* Format
* Apply suggestions from code review
Co-authored-by: Greg Spencer <gspencergoog@users.noreply.github.com>
Co-authored-by: Greg Spencer <gspencergoog@users.noreply.github.com>
This PR adds the dark theme support for both windows and winuwp engine, based on flutter/flutter#54612.
On Win32: using a registry value that is not documented. (I tested on Windows 10.0.19043.1165)
On UWP: using default background color. (I couldn't find WinRT API that returns dark/light directly, without xaml package.)
Flutter PR for win32: flutter/flutter#88520.
This PR will closeflutter/flutter#54612.
This PR will complete one item in flutter/flutter#70214.
