A mistake was introduced in https://github.com/flutter/engine/pull/39498 where the maskViews are already recycles each frame.
Sometimes a PlatformView does not need to be re-composite: (https://github.com/flutter/engine/blob/main/shell/platform/darwin/ios/framework/Source/FlutterPlatformViews.mm#L398-L401), so the mask view for such PlatformView should not be recycled.
This PR changed the `recycleMaskViews` API to allow individual maskviews to be recycled. `ApplyMutator` then only recycle the maskView for that particular PlatformView.
The MaskViewPool is also reworked to be simpler.
- The pool now contains a single set of mask views, there is no index counter needed.
- When a maskView is needed, try to get it from the pool.
- If pool is empty, create a new view.
- If pool has an available maskview, remove it from the pool.
- When a PlatformView starts to `applyMutator`, it removes current the maskView, insert the maskView to the pool.
- When the above PlatformView needs to a maskView, it grabs one from the pool.
fixes: https://github.com/flutter/flutter/issues/125620
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Fixes b/282290672
Google Testing currently does not support vulkan, and constructors like
this need to be behind a flag.
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or Hixie said the PR is test-exempt. See [testing the engine]
for instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.
This optimization avoids recording unnecessary render operations that will not affect the output and also eliminates the need for "draw detection" mechanisms like `DlOpSpy` and `CanvasSpy` by remembering if any non-transparent operations were included. The `DlOpSpy` unit tests were updated to check if the results from that object match the new `DisplayList::affects_transparent_surface()` method.
Fixes https://github.com/flutter/flutter/issues/125338
In addition, this change will unblock some other Issues:
- https://github.com/flutter/flutter/issues/125318
- https://github.com/flutter/flutter/issues/125403
Noticed this while working on https://github.com/flutter/flutter/issues/124181
In Impeller right now, Android creates a context for each surface - but in Skia, Android creates one context and shares it across surfaces. For the multiple-engine scenario this matters. This also makes it possible for Android to provide the impeller::Context independently of surface creation.
## Description
In https://github.com/flutter/engine/pull/41702, the default state of the focus bit is "false", assuming that Android will send an `onWindowFocusChanged(true)` when the window is first focused, but there appear to be some cases where that doesn't happen.
This change puts the initial state back to what it used to be: in the absence of focus change events, entering the "onResume" Android state will report the `resumed` state in Flutter. Before this PR, and after https://github.com/flutter/engine/pull/41702, if no focus events were received, it would default to `inactive`.
## Tests
- Updated tests to match.
These tests are flaky. Until they've been deflaked, disable the
following two tests:
* FlutterEngineTest.BackgroundIsBlack
* FlutterEngineTest.CanOverrideBackgroundColor
Issue: https://github.com/flutter/flutter/issues/124677
The original code creates an internal `class` which is alarming:
`NSClassFromString(@"_UIVisualEffectBackdropView")`
This PR removes such usage.
*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/124008
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
As we move the Dart FIDL bindings we need to stop calling FIDL methods
from Dart in the Flutter repo. This only affects some integration tests.
This moves the FIDL calls into C++ and uses platform messages to make
the functionality available to the Dart tests.
Remove reference to unused fuchsia.sys.Environment protocol in test
Realm setup. Besides being unused,
the reference was not correctly matched by an #include, nor a GN build
dependency.
Bug: fxbug.dev/81285
Fixes https://github.com/flutter/flutter/issues/123307 - for Android, iOS, and Web, for the main display only (https://github.com/flutter/flutter/issues/125938 tracks supporting multiple displays, https://github.com/flutter/flutter/issues/125939 for desktop).
Desktop will need to be implemented for this, but given priority for a couple of our customers targetting foldable devices on Android I'm inclined to get this in before desktop can be finished.
The main concern for this right now is that on some Android foldable devices, setting a preferred orientation will cause letterboxing and the `MediaQuery` will _never_ get the full screen size when unfolded. This causes apps to think the screen is smaller than it is, as they've mainly been using `MediaQueryData.size` to figure this out. Android's recommendation is to not set a preferred orientation, and if you must to use the new method introduced in `ViewUtil.java` to calculate the maximal window size.
Removes the physical model layer and associated engine code. This was already deprecated and removed in the framework. By removing it in the engine, we can also remove the need for layer tree diff/paint/preroll to have the device pixel ratio. This will simplify some of the multi-view work
Fixes https://github.com/flutter/flutter/issues/125720
## Description
This reverts commit e49577708d9d5315fa6c001d7dc20ee80d04cd35 to re-land #41094 because the Google test failures have been fixed. There are no changes to the original PR, since the fixes were in the Google code.
Typically the AndroidShellHolder's apk_asset_provider_ is set by AndroidShellHolder::Launch when the DartExecutor runs the engine's entrypoint.
But if the engine was started by Spawn, then the apk_asset_provider_ was not being set. This would cause a crash if the resulting engine was later used to spawn another engine.
Fixes https://github.com/flutter/flutter/issues/122364
Switching the calls to dispatch into an Impeller Dispatcher to use a cull rect to enable pre-culling of the out-of-bounds ops.
This change showed an improvement of around 2x on the rendering performance of the non-intersecting platform view benchmark, but that was measured without the recent changes to the destructive blend modes in Impeller renderer.
DisposeView happens every frame before laying out PlatformViews, with the order specified in `composition_order_`. Because `view_to_dispose_` is determined on UI thread and `composition_order_` is determined on the platform thread, there could be a race if the dart code on the UI thread runs faster than the rasterizer on the Platform thread, causing a view in `view_to_dispose_` is still in the `composition_order_`.
This PR delays the views in the `composition_order_` being disposed and wait for the next frame to dispose them.
fixes https://github.com/flutter/flutter/issues/125223
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
## Description
This incorporates additional signal from `Activity.onWindowFocusChanged` to help decide if the application is `resumed` or `inactive`.
When the user pulls down the notification shade or opens the app switcher in iOS, then iOS sends a notification to the application that it no longer has input focus (is no longer "active" in Apple terminology).
However, Android (at least on a Pixel) doesn't send `onPause` and `onResume` events for these things, as one might expect. Instead, this PR changes things so that we listen to `Activity.onWindowFocusChanged` and see if any of the windows still have focus.
If it doesn't have focus, then the lifecycle switches to `inactive` (even if `onPause` hasn't been called), and if it does have focus (and `onResume` hasn't been called) then we should go to `resumed`.
State changes are determined and deduped in the `LifecycleChannel` class.
Here's the old state table:
| Android State | Flutter state |
| ------------- | ------------- |
| Resumed | resumed |
| Paused | inactive |
| Stopped | paused |
| Detached | detached |
Here's the new state table:
| Android State | Window focused | Flutter state |
| ------------- | ------------- | ------------- |
| Resumed | true | resumed |
| Resumed | false | _inactive_ * |
| Paused | true | inactive |
| Paused | false | inactive |
| Stopped | true | paused |
| Stopped | false | paused |
| Detached | true | detached |
| Detached | false | detached |
* = This is the relevant change in this PR.
("Window focused" means one or more windows managed by Flutter are focused)
The `inactive` state is for when the application is running and visible, but doesn't have the input focus. An example where this currently happens are when a phone call is in progress on top of the app, or on some OEMs when going into the app switcher (I've tested on Realme and it does that, at least). With the PR, it will also go into `inactive` when the app has lost input focus, but is still in the Android `onResume` state. This means that on phones that don't pause the app when they go into the app switcher or the notification window shade (Pixel, others), the app will go into `inactive` when it didn't before. If developers weren't doing anything special in the `inactive` state before, then this PR will have no change for them. If they were, they will go into that state more often (but more consistently across OEMs).
## Related Issues
- Fixes https://github.com/flutter/flutter/issues/124591
## Tests
- Added unit tests for handling `onWindowFocusChanged`.
Implements partial repaint for Impeller.
Fixes https://github.com/flutter/flutter/issues/124526
The new code that manages the damage regions is more or less a copy paste from the existing Skia implementation. Compared to Skia, there are a few differences:
Normally Impeller wants to use the drawable as the resolve texture for the root MSAA pass. Unfortunately this will unconditonally clear that texture. Thus to do a partial repaint, we have to allocate a separate texture to resolve to and then blit into the drawable.
The blit seems to take about 500ns for a full screen on an iPhone 13. That implies that partial repaint is likely not worth doing if the screen is significantly changed. Thus I've added code in compositor_context.cc that computes the percentage of width or height that is part of the dirty rect. Above a threshold of (abitrarily chosen) 70%, we just render as normal. This should mean there is only a very minor hit from performing the diff on screens that are highly changed.
The other special case, is that sometimes we get damage rects that are empty - that is the drawable is already completely up to date with what we want to render. IN that case I shortcircuit all of the impeller code and just present immediately. I previously tried returning without a present but this resulted in Xcode reporting dropped frames. One caveat here is that if you use the XCode frame debugger and attempt to capture a frame where we early present, then it will claim it couldn't capture any command buffers (because we didn't create any).
To facilitate all of this, I added some additonal plumbing so that the impeller surface can get the clip rect from the submit info. Additionally, rather than using a clip rect impeller will translate and then shrink the root surface texture. This reduces memory usage compared to just clippling.
Reverts https://github.com/flutter/engine/pull/40895.
Resolves https://github.com/flutter/flutter/issues/120399 (again).
A bunch of frames get pumped on the main thread _without a transaction_
just before unmerging occurs (I don't know why this happens), and so
checking the current thread to determine whether we need to present with
a transaction or not isn't sufficient. In the prior fix, after the
unmerge, the raster thread would hang for one second while waiting for
the next drawable to get freed up (happens on the second raster thread
frame post-unmerge), and then subsequent presents would just do nothing
for a while, but eventually recover.
`presentsWithTransaction` works whether the `CATransaction` stack is
empty or not, and so the only difference here is that
`presentsWithTransaction` is always turned on and `presentDrawable` is
always avoided (otherwise it tries to present too early and nothing
renders when platform views are present).
These keep turning up, so I did a little bash-fu to find them all.
```bash
grep -lL "#pragma once" $(grep -lL "#ifndef .*_H_" $(find . | grep "\.h$")) | cut -c 3-
```
No change in functionality. I've just renamed the TU's to replace `display_list_` to `dl_`. I am sure there are chances to make the naming better. For instance, the op receiver still being named a dispatcher.
After having run a memory audit on Flutter Gallery I believe we can turn
on wide gamut support by default for impeller. Here are the
ramifications of this change:
1) If an opaque surface is used and no transparent wide gamut images are
used, there should be no change in performance or memory usage.
1) If a transparent surface is used; the surface will take up 2x memory
(ex 4MB on iPhone 7) and will have a runtime performance cost.
1) If wide gamut **transparent** images are used, they will take up 2x
memory. Opaque wide-gamut images will take up the same amount of memory.
## Flutter gallery test results
### Test 1
Steps:
* Tap "Material"
* Scroll down and back to "Cards"
* Tap "Cards"
* Scroll to bottom of "Cards" example
Notice that the card images are wide gamut, AdobeRGB.
**Memory increase:** 10.401 MB (+8.7%)
### Test 2
Steps:
* Just launch the gallery
Notice that there shouldn't be any wide gamut images.
**Memory Increase:** 1.22 MB (+1%, this is probably a fluke of
measurement)
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or Hixie said the PR is test-exempt. See [testing the engine]
for instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
This change migrates `touch-input` integration tests from the
`gfx-root-presenter-test-ui-stack` UI test realm variant to run
parameterized tests of two types: `gfx-scene-manager-test-ui-stack` and
`flatland-scene-manager-test-ui-stack`. Both are exercised for the
generic tap test, and only GFX is exercised for embedded view cases.
This will enable fuchsia.git to remove the
`gfx-root-presenter-test-ui-stack` variant, which will no longer be
supported.
This change also:
- removes or updates all remaining references of root presenter, which
manifested in the form of inline code comments
- adds a TODO to update the child view app for the embedded touch input
test cases
Note that this change does _not_ modify the `embedder` integration
tests, which use a generic `test-ui-stack` variant as defined by the
fuchsia pkg url:
`fuchsia-pkg://fuchsia.com/test-ui-stack#meta/test-ui-stack.cm`. Since
the contents of this package is defined in fuchsia.git, the realm under
test can adapt to changes in fuchsia.git so long as they update the
test-ui-stack build target (which they do in https://fxrev.dev/831359,
the change that relies on the touch changes in this repo described
above.)
Fixes https://fxbug.dev/125304
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or Hixie said the PR is test-exempt. See [testing the engine]
for instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.
With this PR we no longer need to hold DisplayLists in GPUObject wrappers and they can be disposed instantly instead of queueing on the Unref thread.
This will definitely be a win for Impeller as none of the objects used in a frame now require queueing, but the performance impact on apps running on top of skia is less clear if they depend on a lot of images inside their DisplayLists that still need to be queued to be freed. After getting further in the work, it looks like only decoded images need to use the protected DlImage wrappers and most of those should survive many frames before they are disposed. That should hopefully leave very few unrefs happening per frame.
~There are 3 unit tests in `shell_unittests.cc` and `embedder_metal_unittests.mm` that are now GSKIP'd as they now invoke code that needs a fully initialized UIDartState in order to protect their images. I will look into fixing the tests and/or making the code they invoke provide protection without relying on UIDartState.~ (This looks to be fixed in the latest commit by simply not creating DlImageGPUs all over the source base and simply catching only those that end up in UI data structures. There is actually existing code in one of the modules that feeds ui.Image with an answer to wrap the image in a DlImageGPU if it has a skia image anyway, so most of these additional uses of DlImageGPU that were having trouble getting the Skia unref queue just didn't need it anyway.)
