Delegate a11y events and action to/from embedded Android platfrom views.
This handles delegation of:
* AccessibilityNodeProvider#performAction
* ViewGroup#requestSendAccessibilityEvent
* View#onHoverEvent
Additionally updates the currently input accessibility focused node state that is
tracked by the a11y bridge when an embedded view's node is focused.
All embedder unit-tests have to setup the Flutter project arguments from scratch
before launching the engine. The boilerplate and having to deal with the low
level C API during each engine launch is a hinderance to writing tests.
This patch introduces an EmbedderTest fixture that sets up all the embedder side snapshots before allowing the unit test to create a FlutterConfigBuilder` that
the test can use to incrementally build and edit the Flutter project
configuration. From the given state state of a configuration, multiple engines
can be launched with their lifecylces managed by appropriate RAII wrappers.
This allows the a fully configured Flutter engine to be launched using 4 lines
of code in a fixture.
```
EmbedderConfigBuilder builder;
builder.SetSoftwareRendererConfig();
builder.SetAssetsPathFromFixture(this);
builder.SetSnapshotsFromFixture(this);
auto engine = builder.LaunchEngine();
```
An activity can use ViewFactory.retainNativeFlutterView to reuse a
FlutterNativeView across multiple instances of the activity. In this
scenario, the FlutterNativeView should continue to handle incoming messages
sent from Dart.
This PR mirrors virtual a11y tree of embedded platform views in the Flutter
a11y tree.
Non virtual hierarchies are not currently supported.
Only works on Android versions earlier than Android P as it relies on
reflection access to hidden system APIs which cannot be done starting
Android P.
A11y is not yet working as we also need to delegate a11y events from the
platform view to the FlutterView. This will be done in a following PR to
keep the change size a little saner.
Changes include:
- File structure
- Header guards
- Include paths
- Namespaces
- Integration with the engine's GN build
- Conversion from jsoncpp to rapidjson
- Style and clang-format adjustment to match engine repository
PlatformViewsDelegate.
Since onDetachedFromWindow can be called after the activity was
destroyed, the previous call to detach the accessibility bridge could
have crash as the NativeFlutterView was already null.
Layout occurs after [CATransaction commit]. layoutSubviews was where we set the contentsScale on the CALayer. This meant that for one frame, we would see content on the overlay view which was did not have the correct content scale.
This change makes it so that we initialize the FlutterOverlayView with the correct contentsScale.
This also updates the overlay_gr_context_ when we first create the overlay_view. This is an artifact of #8175.
This manifests as jank as seen in: flutter/flutter#29573
Optimize the performance of the platform view composition.
When recomposition is required, we previously remove all the platform view related views (intercepting view, platform view) and re-add them for each frame. This is mentioned in flutter/flutter#23793.
This PR enhanced the performance by re-arrange the existing UIViews and only add the newly created Views.
As a result, it would also fixflutter/flutter#29427
The reason we didn't merge just the gpu and platform threads from the get go was a deadlock in Shell:OnPlatformViewCreated and Shell:OnPlatformViewDestroyed.
The deadlock was caused by the platform thread starting a thread-hopping flow that ends ends up with the gpu thread releasing a latch that the platform thread is waiting on just after starting the cross-thread dance.
If the platform and gpu threads are the same, that last task that is posted to the gpu thread will never get executed as the gpu/platform thread is blocked on a latch.
This works around the deadlock by having a special case in the code for the scenario where the gpu and platform threads are the same.
Fixes: flutter/flutter#23974
There can be cases where SubmitFrame gets called
before overlays are a part of the frame, in these
cases, we should not update the GRContext ahead of time.
This commit makes it so we will update it only when
the frame really shows the overlay.
This addresses: https://github.com/flutter/flutter/issues/28920
This will allow us to easily visualize the time the platform informed the engine of a vsync event, its arguments, and when the engine began its UI thread workload using this information.
Verified that the tests fail on issues like https://github.com/flutter/engine/pull/8166. Unfortunately, there is no x-platform way to perform this check but this should gate incorrect traces being added to the engine.
Allow Flutter to automatically dump the skp that triggers new shader compilations. This is useful for writing custom ShaderWarmUp to reduce jank. By default, it's not enabled to reduce the overhead. This is only available in profile or debug build.
Later, we can add service protocol support to pull the skp from the client to the host. Currently, it works fine for Android-based devices (including our urgent internal clients) where we can `adb shell` into the cache directory.
The ICU data is packaged within libflutter.so, and the engine uses
dlsym(RTLD_DEFAULT) to look up the ICU data symbols in the libraries
loaded by the process.
This approach apparently fails on some Xiaomi devices. If RTLD_DEFAULT
can not find the symbols, then the platform will pass the path to the
libflutter.so library so the engine can use that to resolve the symbols.
Fixes https://github.com/flutter/flutter/issues/28208
Some embedders may have to wait on fences asynchronously before committing
contents. This allows them to post a task onto the engine managed thread used
for rendering.
Some clients (e.g., embedded devices) prefer generating persistent cache files for the specific device beforehand, and ship them as readonly files in OTA packages.
This updates the FlutterSemanticsAction enumerator identifiers for the
'move cursor forward/back one word' actions (added in
flutter/engine#8033) for consistency with the 'move cusor forward/back
on character' identifiers.
ABI compatibility is unaffected, but this with require the following
change in any embedder making use of these fields:
Rename:
kFlutterSemanticsActionMoveCursorForwardByWordIndex
to:
kFlutterSemanticsActionMoveCursorForwardByWord
Rename:
kFlutterSemanticsActionMoveCursorBackwardByWordIndex
to:
kFlutterSemanticsActionMoveCursorBackwardByWord
The current assumption is that the embedder will wait till the vsync event and
then fire the callback. However, some embedders have that information upfront.
Since the time point has already been specified by the embedder, there is no
reason to burden the embedder with having to setup a wait either.
Adds scroll event handling to the macOS shell, sending them using the
new embedding API.
Currently, trackpad scrolling is just handled like scrollwheel
scrolling, since the trackpad-gesture-based scroll handling in Flutter
is still in progress.
Adds code to synthesize Add events if mouse tracking isn't enabled,
so that it doesn't rely on Flutter being lenient about handling events
from pointers that have never been added.
