Remove pending keyboard event ID and use the object directly.
Use references to make sure FlKeyboardManagerData is freed.
Refactor marking events as handled
This is pretty tricky bug - I suspect that because Apple's internal recognizer caching an outdated state of our delaying recognizer.
The conflict happens between WebKit and platform view's recognizers. It happens to all plugins that uses a WKWebView, or any view that has a similar (unknown) gesture setup.
This fix has to be in the engine (rather than the plugin), because the plugin itself knows nothing about the existence of our delaying recognizer.
Here are the steps of my research for future reference:
1. The bug only happens when the overlay flutter widgets blocks the gestures for the platform view (e.g. tap on the platform view area when a flutter context menu is displayed). When the bug happens, WKWebView's link doesn't work anymore, however, the link can still be highlighted when tapped.
2. When I remove the delaying recognizer from platform view, the link became clickable again. This means that the bug is related to some conflict between WKWebView's internal recognizers and our delaying recognizer. This also means that it is not possible to fix this issue at plugin level, which knows nothing about the delaying recognizer.
3. When we tap on the web view when context menu is displayed, `blockGesture` will be called, which simply toggles delaying recognizer's state from `possible` to `ended` state, meaning it should block all recognizers from the current gesture (and it correctly did so). Then I use `dispatch_async` and check the state again, and confirmed the state is correctly reset to `possible` state.
4. Subsequent tap on web view triggers `acceptGesture`, which turns the `possible` state into `failed` state. This subsequent tap only highlights the link, but not activate the link. This suggests that some internal web kit recognizer that handles the highlight sees the `failed` state of delaying recognizer (which is correct), but the recognizer that handles the link activation probably sees the stale state of `possible` or `ended` (which is outdated old state).
5. So the solution is trying to make the recognizer "see" the updated state rather than the cached old state.
6. I tried recreating a new delaying recognizer when `blockGesture` is called:
```
- blockGesture {
delayingRecognizer.state = .ended
dispatch_async {
// force re-create the delaying recognizer
}
}
```
This fixed the link activation bug, however, when opening the context menu again, the gesture is not blocked anymore. This means web kit internal recognizers likely cache the old delaying recognizer for state update, thus the new instance of delaying recognizer won't work anymore. So we can't change the instance. However, it's a good experiment that confirms my hypothesis that some internal webkit recognizer caches the outdated state of delaying recognizer.
7. For the above code, rather than using `dispatch_async`, I also tried `dispatch_after`, and it turns out that it only works if the dispatch_after delay is `0` - even if the delay is much smaller than 1 frame's time (16.7ms), it doesn't work. This means the state checking happens either at the end of the current run loop, or beginning of the next run loop. (not too important information, but it helps me better understand how UIKit works).
8. So from 6, we know that we have to keep the original instance of delaying recognizer. I tried toggling `recognizer.enabled`, it didn't work. I also tried inserting a dummy recognizer, it didn't work. Neither approach triggers the state "refresh" for those webkit internal recognizers.
9. I tried removing and adding back the delaying recognizer, and it just worked! This means that removing and adding back the delaying recognizer probably triggered UIKit to refresh the states for all its related recognizers (i.e. those recognizers either blocking or being blocked by the delaying recognizer), hence getting the updated state.
*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/158961
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
In the case where `CreateImpellerContext` encounters an error while creating an `impeller::ContextMTL`, we logged an error, returned nullptr, then immediately dereferenced the null pointer. Now, rather than crash due to a segfault, we now intentionally abort with an appropriate error message.
This adds checks in the `FlutterDarwinContextMetalImpeller` initialiser that aborts with an appropriate error message if impeller context creation fails, Metal device creation fails, or texture cache creation fails.
Rather than bailing out and returning nil from the initialiser to pass the buck to the caller, we terminate since without a graphics context, the app won't be able to render anything to begin with.
Issue: https://github.com/flutter/flutter/issues/157489
Issue: [b/378790930](http://b/378790930)
No test changes since this just changes an accidental crash to an intentional crash.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Split the channel messaging out of the system channels to make them
simpler to understand and refactor.
The new channel classes could be automatically generated in a second
phase, e.g. using Pigeon to reduce code usage.
The new classes don't have tests as they will already be covered by the
existing code.
The previous mock required knowing the specific functions used in the
binary messenger, this method instead allows test code to provide
complete platform channel implementation for testing and make simulated
platform channel calls into embedder code.
Incorrect reference counting of GTask objects meant platform channel
method calls would leave tasks alive that would leak memory and leave
unclosed references to the binary messenger.
The same problem with NVIDIA drivers which causes issue [152099](https://github.com/flutter/flutter/issues/152099) occurs with Vivante Corporation drivers.
Quick fix for issue on Vivante drivers:
https://github.com/flutter/flutter/issues/152099
- [ x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
- [ x] I read the [Tree Hygiene] wiki page, which explains my responsibilities.
- [ x] I read and followed the [Flutter Style Guide] and the [C++, Objective-C, Java style guides].
- [ x] I listed at least one issue that this PR fixes in the description above.
- [ x] I added new tests to check the change I am making or feature I am adding, or the PR is [test-exempt]. See [testing the engine] for instructions on writing and running engine tests.
- [ x] I updated/added relevant documentation (doc comments with `///`).
- [ x] I signed the [CLA].
- [ x] All existing and new tests are passing.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Applies non-null by default annotations to
FlutterPlatformViewsController with opt-outs where necessary. Updates unit tests to create graphics contexts where requried.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This is a minor refactoring that moves a global bool to a boolean ivar in FlutterPlatformViewsController. The purpose of this variable is simply to avoid the overhead of trying to create backdrop filters if we've ever failed to create one in the past.
Given that this class will only ever have the one instance created and held by per engine with the same duration, and that most apps only ever have one engine, the performance win will be identical for most apps. For the few add-to-app cases with multiple engines either at once or over the course of an app's lifetime, the costs associated with firing up an engine are already a far bigger hit than those being saved by this bool.
Also migrates from C++ style namespace { ... } to Obj-C style static functions. These are entirely equivalent as both restrict symbols to the current translation unit.
This is a reland of https://github.com/flutter/engine/pull/56805, which was reverted as part of https://github.com/flutter/engine/pull/56817.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This migrates PlatformViewController from C++ to Objective-C. Generally, we try to keep the embedder interfaces and components written in Objective-C except for the few places where C++ interfaces are requried to interface with engine APIs such as Shell and PlatformView (e.g. the PlatformViewIOS subclass). Now that the implementation is Objective-C, the class and file are renamed to match Objective-C naming conventions.
This allows us to take advantage of ARC and weak references, which eliminates the need for std::shared_ptr, fml::WeakPtr etc. Further, this eliminates some particularly unintuitive behaviour wherein this class was owned via a std::shared_ptr held by FlutterEngine, and injected into many other classes (e.g. AccessibilityBridge) via a std::shared_ptr& reference -- such that only one instance of the std::shared_ptr actually ever existed, presumably to avoid std::shared_ptr refcounting overhead. Given that this overhead was only incurred a single time at engine initialisation, this seems like overkill. One might ask why it wasn't therefore held in a `std::unique_ptr` and a `std::unique_ptr&` reference passed around. Likely, this was because we wanted to take a `fml::WeakPtr` reference on it.
Regardless, none of this is necessary any longer now that we can inject `__weak FlutterPlatformViewsController*` instances to classes that use it.
To be clear, this patch makes no attempt whatsoever to simplify or clean up the interface or implementation of this class. This class ties together far too many concepts and is injected into far too many places, and we should break it up and simplify it. However, the goal of this patch was simply to port to an Objective-C interface that plays nicely with the rest of the iOS embedder. This does include a couple minor cleanups in `#include`/`#import` order and usage to match our style guide.
This is a reland with a one-line fix of a lambda-capture block to ensure `self` and any local variables are captured by value rather than by reference:
* In the case where this method is called on the platform thread (i.e. where the UI and platform thread are merged), we use the latch to pause the calling thread until the lambda completes, in which case all locals could be passed by reference since the locals are guaranteed to hang around until the lambda completes and signals the latch.
* In the case where this method is called from the UI thread (i.e. where UI and platform thread are not merged), locals may have gone out of scope by the time the lambda executes, leading to undefined behaviour if passed by reference; thus we always pass by value to be sure; since `latch` must be shared between threads, it's passed held in a `std::shared_ptr` so the underlying latch/mutex is shared but it's kept live until it goes out of scope in both threads.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
While recently updating the DlColorSource sources I noticed some questionably implementation choices in the Color variant of the color sources.
I then realized that there was no public use of these classes (other than mostly their own unit tests) and so they should be deleted to focus on implementing the variants that are actually used by Flutter.
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
Unlike https://github.com/flutter/engine/pull/56738, does not touch the vsync API - which looks like it depends on scheduling behavior today, at least for iOS.
This is a combination of 3 reverts, required to get back to the revert that caused `ios_platform_view_tests` to start failing in the framework repo. In reverse chronological order, this reverts two trivial commits plus the non-trivial commit that likely caused the breakage:
* Revert "iOS: Eliminate global in platformviews controller (#56805)" This reverts commit cea4600caa7098fa7ec109d18b869db46cda726a.
* Revert "iOS: Delete FlutterPlatformViewsController.layerPoolSize (#56806)" This reverts commit 80fa8a590876e0d29055b9ddbfa8670c1f83759e.
* Revert "iOS: Migrate PlatformViewsController to Objective-C (#56790)" This reverts commit afd05afc406deb79fbe9c16684aeeeb19322b288.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Objective-C files should be named to match their header, in this case, //flutter/shell/platform/darwin/ios/framework/Headers/FlutterPlatformViews.h.
When private API is required, we often create a header named FlutterFoo_Internal.h, but the implementation file is always just FlutterFoo.mm. This brings FlutterPlatformViews back into line with convention.
No test changes since this is just a file rename.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This is a minor refactoring that moves a global bool to a boolean ivar in FlutterPlatformViewsController. The purpose of this variable is simply to avoid the overhead of trying to create backdrop filters if we've ever failed to create one in the past.
Given that this class will only ever have the one instance created and held by per engine with the same duration, and that most apps only ever have one engine, the performance win will be identical for most apps. For the few add-to-app cases with multiple engines either at once or over the course of an app's lifetime, the costs associated with firing up an engine are already a far bigger hit than those being saved by this bool.
Also migrates from C++ style namespace { ... } to Obj-C style static functions. These are entirely equivalent as both restrict symbols to the current translation unit.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This migrates PlatformViewController from C++ to Objective-C. Generally, we try to keep the embedder interfaces and components written in Objective-C except for the few places where C++ interfaces are requried to interface with engine APIs such as Shell and PlatformView (e.g. the PlatformViewIOS subclass). Now that the implementation is Objective-C, the class and file are renamed to match Objective-C naming conventions.
This allows us to take advantage of ARC and weak references, which eliminates the need for std::shared_ptr, fml::WeakPtr etc. Further, this eliminates some particularly unintuitive behaviour wherein this class was owned via a std::shared_ptr held by FlutterEngine, and injected into many other classes (e.g. AccessibilityBridge) via a std::shared_ptr& reference -- such that only one instance of the std::shared_ptr actually ever existed, presumably to avoid std::shared_ptr refcounting overhead. Given that this overhead was only incurred a single time at engine initialisation, this seems like overkill. One might ask why it wasn't therefore held in a `std::unique_ptr` and a `std::unique_ptr&` reference passed around. Likely, this was because we wanted to take a `fml::WeakPtr` reference on it.
Regardless, none of this is necessary any longer now that we can inject `__weak FlutterPlatformViewsController*` instances to classes that use it.
To be clear, this patch makes no attempt whatsoever to simplify or clean up the interface or implementation of this class. This class ties together far too many concepts and is injected into far too many places, and we should break it up and simplify it. However, the goal of this patch was simply to port to an Objective-C interface that plays nicely with the rest of the iOS embedder. This does include a couple minor cleanups in `#include`/`#import` order and usage to match our style guide.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Reverts: flutter/engine#56606
Initiated by: LongCatIsLooong
Reason for reverting: https://github.com/flutter/flutter/issues/159456
Original PR Author: LongCatIsLooong
Reviewed By: {chunhtai, cbracken}
This change reverts the following previous change:
This PR adds basic FKA scrolling support: when the iOS focus (the focus state is maintained separately from the framework focus, see the previous PR) switches to an item in a scrollable container that is too close to the edge of the viewport, the container will scroll to make sure the next item is visible.
Previous PR for context: https://github.com/flutter/engine/pull/55964https://github.com/user-attachments/assets/84ae5153-f955-4d23-9901-ce942c0e98ac
### Why the UIScrollView subclass in the focus hierarchy
The iOS focus system does not provide an API that allows apps to notify it of focus highlight changes. So if we were to keep using the transforms sent by the framework as-is and not introducing any UIViews in the focus hierarchy, the focus highlight will be positioned at the wrong location after scrolling (via FKA or via framework). That does not seem to be part of the public API and the focus system seems to only know how to properly highlight focusable UIViews.
### Things that currently may not work
1. Nested scroll views (have not tried to verify)
The `UIScrollView`s are always subviews of the `FlutterView`. If there are nested scrollables the focus system may not be able to properly determine the focus hierarchy (in theory the iOS focus system should never depend on `UIView.parentView` but I haven't tried to verify that).
2. If the next item is too far below the bottom of the screen and there is a tab bar with focusable items, the focus will be transferred to tab bar instead of the next item in the list
Video demo (as you can see the scrolling is really finicky):
https://github.com/user-attachments/assets/51c2bfe4-d7b3-4614-aa49-4256214f8978
I've tried doing the same thing using a `UITableView` with similar configurations but it seems to have the same problem. I'll try to dig a bit deeper into this and see if there's a workaround.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
In flutter/engine#35501, handling was added to log a debug message to the console in the case where a platform view with a non-zero origin was identified.
Unfortunately:
* In unopt builds, the first thing we do in that block is to call FML_DCHECK asserting that the origin is zero, so we never actually emit the log statement.
* In opt builds, FML_DCHECK is a no-op, so users are unlikely to actually ever notice the crash.
The proper fix is to eliminate this restriction, but in the meantime, this eliminates this block entirely and leaves the TODO. We've had only two comments on that bug in the 2.5 years since it was added.
Issue: https://github.com/flutter/flutter/issues/109700
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This PR adds basic FKA scrolling support: when the iOS focus (the focus state is maintained separately from the framework focus, see the previous PR) switches to an item in a scrollable container that is too close to the edge of the viewport, the container will scroll to make sure the next item is visible.
Previous PR for context: https://github.com/flutter/engine/pull/55964https://github.com/user-attachments/assets/84ae5153-f955-4d23-9901-ce942c0e98ac
### Why the UIScrollView subclass in the focus hierarchy
The iOS focus system does not provide an API that allows apps to notify it of focus highlight changes. So if we were to keep using the transforms sent by the framework as-is and not introducing any UIViews in the focus hierarchy, the focus highlight will be positioned at the wrong location after scrolling (via FKA or via framework). That does not seem to be part of the public API and the focus system seems to only know how to properly highlight focusable UIViews.
### Things that currently may not work
1. Nested scroll views (have not tried to verify)
The `UIScrollView`s are always subviews of the `FlutterView`. If there are nested scrollables the focus system may not be able to properly determine the focus hierarchy (in theory the iOS focus system should never depend on `UIView.parentView` but I haven't tried to verify that).
2. If the next item is too far below the bottom of the screen and there is a tab bar with focusable items, the focus will be transferred to tab bar instead of the next item in the list
Video demo (as you can see the scrolling is really finicky):
https://github.com/user-attachments/assets/51c2bfe4-d7b3-4614-aa49-4256214f8978
I've tried doing the same thing using a `UITableView` with similar configurations but it seems to have the same problem. I'll try to dig a bit deeper into this and see if there's a workaround.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Bumps the version of AGP used in the IDE-support `build.gradle`, as well as the robolectric version in both the IDE-support `build.gradle` and test-runner-`build.gradle`.
This is the current latest robolectric: https://github.com/robolectric/robolectric/releases/tag/robolectric-4.14.1.
Also
1. configures robolectric to use API 35, and
2. removes the use of a deprecated class which (from what I could tell) looked like it was just used for setup, and the test still passes without it.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Reverts: flutter/engine#56738
Initiated by: jonahwilliams
Reason for reverting: speculative revert for framework failures.
Original PR Author: jonahwilliams
Reviewed By: {jason-simmons}
This change reverts the following previous change:
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
This also updates the vsync waiter to use RunNowOrPostTask, so that we start vsync events as early as possible.
The Skia snapshot controller will activate the delegate's surface render context on the current thread. If the delegate has no surface, then it will use the snapshot surface producer to create a temporary surface.
The Impeller snapshot controller needs to do the same in order to support OpenGL/GLES scenarios where the thread does not currently have an EGL context.
The DlColorSource code uses Skia geometry classes for its internal computations. This PR switches those implementations to use the Impeller geometry classes for consistency and 3rd party header file independence.
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
This also updates the vsync waiter to use RunNowOrPostTask, so that we start vsync events as early as possible.
Moves code specific to each graphics backend into the (existing) translation unit associated with that backend.
Previously, we could not include any Objective-C types in `shell_test_platform_view_metal.h`, since that file was included into `shell_test_platform_view.cc`, which is pure C++. To work around this, we had encapsulated Objective-C Metal types in a `DarwinContextMetal` struct hidden in the implementation file with a pointer to it forward-declared in the header.
We now use Metal types directly in the header, without the workarounds.
Issue: https://github.com/flutter/flutter/issues/158998
Issue: https://github.com/flutter/flutter/issues/137801
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
`kDefaultBackendType` is intended to make life easier for authors of tests, but in any switch statement where it's used (currently just a single location), we rely on ordering it first and `#ifdef`ing out all backends that aren't available.
Instead, we define a static function that returns the default that callers can invoke instead. This avoids relinace on case ordering and fallthrough.
In https://github.com/flutter/engine/pull/56722 we split backends out into separate translation units, and ideally should remove the `#ifdef`s, which means we can't rely on this trick anymore.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The Info.plist override was missed earlier.
The main changes to the tests are due to the UI thread merge with the platform thread. The amendment to the "spawn" API are due to the fact that the assertions checked for the presence of GrDirectContext which will never exist with Impeller.
Unblocks https://github.com/flutter/engine/pull/56706
This PR introduces a `NativeAssetsManifest.json` next to the `AssetManifest.json` and `FontManifest.json`. This removes the need for embedding the native assets mapping inside the kernel file and will enable decoupling native assets building and bundling from the kernel compilation in flutter tools. This will then allow us to remove dry-run from the build hook protocol.
(It also means all isolate groups will have the same native assets. However, since Flutter does not support `Isolate.spawnUri` from kernel files anyways, this is not a regression.)
This manifest is parsed eagerly on startup by the engine in a manner similar to how the font manifest is parsed. The manifest contents need to be available in the callback for resolving assets, which does not have access to the engine. Therefore the parsed manifest is `NativeAssetsManager` stored in the `IsolateGroupData`. The engine passes it in on isolate group creation, and the FFI callbacks access it from the isolate group data.
Issue:
* https://github.com/flutter/flutter/issues/154425
Related PRs:
* https://dart-review.googlesource.com/c/sdk/+/388161
Follow up work:
* This PR does not yet remove the engine callbacks registered via the dart_api that rely on kernel embedding. If we were to do that in this PR, it would require a manual roll of the engine into flutter/flutter with the PR that switches flutter_tools to emit the native assets manifest instead of embedding in kernel, and a manual roll into g3 to switch emitting a manifest instead of embedding in kernel. A TODO is left in the code for those callbacks to be removed.
## Testing
Most of this PR cannot be tested in isolation. The code in this PR is heavily exercised in the follow up flutter_tools PR which creates the `NativeAssetsManifest.json` and removes the embedding of native assets in kernel files.
* This PR adds a unit test for parsing the JSON manifest.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The DlImageFilter code uses Skia geometry classes for its internal computations. This PR switches those implementations to use the Impeller geometry classes for consistency and 3rd party header file independence.
Split the channel messaging out of the system channels to make them
simpler to understand and refactor.
The new channel classes could be automatically generated in a second
phase, e.g. using Pigeon to reduce code usage.
The new classes don't have tests as they will already be covered by the
existing code.
In many embedder tests, we want to get at the appropriate backend-specific `EmbedderTestContext` subclass (`EmbedderTestContextGL`, etc.) in order to make backend-specific setup calls such as `SetGLFBOCallback()` or others. Formerly, this required casting the returned `EmbedderTestContext&` to the appropriate subclass in each test.
This templatises the `GetEmbedderContext()` method to return the appropriate backend-specific subclass directly.
Issue: https://github.com/flutter/flutter/issues/158998
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
