Introduces a mechanism to allow backdrop filters to 1) share backdrop inputs and 2) fuse filter applications for faster blurs.
This is a proposed solution to https://github.com/flutter/flutter/issues/131568
Implemented:
* Developer can specify a "backdrop id" which indicates that a backdrop layer should share the input texture and potentially cached filter for a layer.
* Removes second save layer for each backdrop filter
* Removes save layer trace event for backdrop filter
* Can fuse backdrop filters if there is more than one identical filter
TBD:
* Adjust heruristic to avoid applying bdf filter to entire screen
Suggestions: applying a bdf should be a distinct operation from a save layer in the DL builder/dispatcher. The saveLayer implmenentation in the impeller dispatcher is super convoluted because it needs to handle both.
### Video
Video starts with normal bdf then I hot reload to specify that the bdfs share inputs/filters. This is running on a pixel 8 pro
Change to the macrobenchmark app is just:
```dart
Widget build(BuildContext context) {
Widget addBlur(Widget child, bool shouldBlur) {
if (shouldBlur) {
return ClipRect(
child: BackdropFilter(
filter: ImageFilter.blur(sigmaX: 5, sigmaY: 5),
backdropId: 1, // Added ID
child: child,
),
);
} else {
return child;
}
}
```
https://github.com/user-attachments/assets/22707f97-5825-43f1-91b4-1a02a43437f5
Requires framework changes in https://github.com/jonahwilliams/flutter/pull/new/backdrop_id
The MockCanvas API was originally used to test the output of rendering methods to match them against a reference set of "MockData" structures. This API has been superseded by just using a DisplayListBuilder to record reference operations and then to compare the resulting DisplayList objects.
This PR deletes the last remaining uses of MockCanvas as well as the mechanism itself.
This is the beginning of the bulk of de-skia-fication work in the engine. All of the standard types in the DlCanvas API now have overloads that specify the corresponding Dl type, mainly for Dl*Rect and DlPoint types. This enables further work to switch from SkFoo types to DlFoo types in the various engine modules culminating in the elimination of the old methods that use the Sk types.
All of the former methods that used the basic Sk types are now implemented as inlinable translation overloads and the underlying implementations of DlCanvas now implement only the newer style interfaces so that they don't need to be further modified as we eliminate the old Skia types from the interface.
There are still a couple of Skia types remaining in the DlCanvas API without any DL type variants which will be handled in a future phase:
- SkRRect
- SkRSXform
- SkTextBlob (will be hidden behind a common interface along with TextFrame)
- SkImageInfo (only used in a few calling sites)
Aiks has been gutted, and most of the reming types need to be merged with dl/entity types and deleted. this is difficult to do with impeller/display_list and aiks as separate gn targets as it requires circular deps.
Lets merge them and remove the aiks directory.
ui.Canvas and ui.SceneBuilder now use the DlPath object directly from the ui.Path object. This results in increased sharing of the wrapper objects which then increases the sharing of both the converted Impeller paths and Skia's volatile flag.
The VolatilePathTracker mechanism is deleted and rather than count the number of frames that a path is stable for, instead we count the number of times it is used for rendering. If a path is used 100 times in a single frame, it will become non-volatile and start being cached almost immediately. The cached Impeller paths are now also tracked for all instances of the same path, rather than for each call site that originated from a DisplayList dispatch.
The `SkRect::intersect` and `SkIRect::intersect` methods return values that must be handled or the calling code might end up with a non-empty answer for non-intersecting rectangles. There were 3 or 4 places in our code that we weren't doing this so now we check them all, in some cases even if we believe that the answer will not be empty.
This is prep work so that Skia can add `[[nodiscard]]` directives to these 2 methods so that nobody else makes that mistake.
Remove the need to merge raster and platform thread in the presence of platform views by defering UIView mutation and presentation of flutter views into separate platform thread task. Fixes priority inversion problem cause by platform thread blocking on drawable aquisition.
Open questions:
* Fixed ~What is a better interface for handling the partial submit with impeller. (TBD)~
* Fixed ~Update: We Don't | How do we fix this for Skia~ Fixed
* Fixed ~Update: Done, we post a task to the platform thread. Is there a shorter term solution for creating overlay layers on the raster thread.~ Fixed
* Fixed ~Update: seems to. Does this perform well enough (independent of platform/ui thread merge and w/ thread merge).~ Fixed
Fixes https://github.com/flutter/flutter/issues/142841
part of https://github.com/flutter/flutter/issues/150525
Reverts: flutter/engine#54219
Initiated by: jonahwilliams
Reason for reverting: incorrect rendering on last frame after platform view is removed.
Original PR Author: knopp
Reviewed By: {jonahwilliams}
This change reverts the following previous change:
Currently we force full repaint when thread merging is detected, but that is not a good way to determine whether platform view is in hierarchy when thread merging is no longer needed. Alternative solution here makes the PlatformViewLayer force full repaint when diffing.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This removes support for "unobstructed platform views" on iOS - instead prefering to use the Android strategy of minimizing overlay layers, as this is generally more performant.
Fixes https://github.com/flutter/flutter/issues/151274
This is only an issue on iOS (so far) because of the platform view rendering strategy that involves submitting multiple impeller frames per compositor frame.
FYI @bdero
This mechanism pulls some clip-reduction logic from Impeller Entities up into DisplayListBuilder to simplify the work that Impeller has to do and also amortize the work if the DisplayList is reused from frame to frame. Since the DisplayList does not have access to information such as surface sizes, there will still be cases that Impeller can catch that must be conservatively included by the recording process in the Builder, so this mechanism does not replace the same code in Impeller, it merely catches some cases earlier, while recording widget output, to avoid the same work on every frame down in Impeller.
In this first pass only the most conservative and straightforward cases are handled - a clip on a layer which has a previous clip that the new clip fully contains (and which was not already restored).
This limited approach is already enough to eliminate a couple of clip operations in the layout of many of the pages in the `new_gallery` benchmarks.
The MatrixClipTracker was nothing more than a vector of MatrixClipState objects and a whole lot of duplicate delegation methods. It provided little value since nearly every use of it already had a state stack that was kept in synch with it's internal stack and so it was just adding an extra layer of delegation and extra allocations.
The size of the LTO build of the engine with the dylib compressed is as follows:
```sh
$ du -k libFlutter*
5236 libFlutter.dylib.tar.gz
4324 libFlutterSlimpeller.dylib.tar.gz
```
Sizes are in KiB. This represents a binary size reduction of 17.41% of the compressed artifacts. The compression ratios will likely differ based on the compression scheme.
Uncompressed, the sizes are:
```sh
$ du -k libFlutter*
16920 libFlutter.dylib
14044 libFlutterSlimpeller.dylib
```
This represents a binary size reduction of 16.99% which is in the same ballpark.
The really mucky bit was backing out the raster cache and persistent cache. I want to clean that up in a later patch so that those TUs are part of a separate submodule.
Opting out of Impeller will lead to a fatal log at startup saying the opt-out is disallowed.
Fixes https://github.com/flutter/flutter/issues/126606
The engine has ostream conversions for most Skia objects, but none of the test files include the files where they are defined. Adding the include file to the `layer_test.h` file will include them on any file which does unit testing on the engine layers.
The `DisplayListMatrixClipTracker` class performs the working of maintaining a stack of cull rects and transforms for classes in the engine that require such functionality (such as `DisplayListBuilder`, `DiffContext`, and `LayerStateStack`.
Until now it has been using a mixture of SkMatrix and SkM44 along with an SkRect to maintain this information, but that complexity involves overhead and implementation complexity which complicates future work on these areas.
By reimplementing the class to use Impeller geometry objects, the internal architecture is greatly simplified allowing for easier future maintenance.
With this PR, backing stores are labeled with view IDs. When the engine requests the embedder to create a backing store, the engine will promise that it will only be used for a specific view.
This follows the design doc http://flutter.dev/go/backing-stores-for-multi-view-partial-repaint, so that backing stores can be used as a front surface that retains its content last frame.
The engine will create a render target cache for each view to cache backing stores separately.
### Alternative design
The separate render target cache for each view is not needed to implement the design doc, since all usages described in the design doc avoids the engine cache. Instead, we can make the engine still only manage one render target cache for all views, and backing stores in it are interchangeable across views. We might describe the behavior in this way:
* In general, the view ID is just provided to the creating callback as information. (This is how it is seen when the engine cache is avoided.)
* If the engine cache is used, then the created backing store might not be immediately collected, and might be reused for different views.
But it's really hard to explain the mechanism and the result is really confusing (as can be seen from my attempt). Why is there a view ID but it's not used, and if you enable the engine cache it's not even followed?
That's why I chose the current approach. Feel free to suggest otherwise for this.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This PR connects the view ID provided by the rasterizer to `EmbedderExternalViewEmbedder`'s presenting call (see `embedder_external_view_embedder.cc`).
This PR also contains a refactor (which actually takes the majority of LOC) that moves the specification of view ID from `PrepareFlutterView` to `SubmitFlutterView`. The `flutter_view_id` parameter in `PrepareFlutterView` was added in https://github.com/flutter/engine/pull/46169. It turns out that we don't need the view ID throughout the preparation phase, so we can delay the specification to submission, which makes it feel cleaner.
### Tests
Existing tests are changed to verify that `SubmitFlutterView` receive the correct view ID.
The feature that `EmbedderExternalViewEmbedder` sends the view ID to the presenting API is not tested for now, because `EmbedderExternalViewEmbedder` is typically tested in `embedder_unittests`, but it requires the embedder API `AddView`. It will be tested in later changes to `EmbedderExternalViewEmbedder`.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The previous PR https://github.com/flutter/flutter/issues/143420 rounds out the layers and rounds in the platform views. This results in missing pixel on the edge of the intersection when there's fractional coordinate (as shown in the screenshot below), because platform view is below the layers.
It turns out that we have to round out both platform view and layers, because:
- rounding in platform view rects will result in missing pixels on the edge of the intersection.
- rounding in layer rects will result in missing pixels on the edge of the layer that's on top of the platform view.
This PR simply skips the single (or partial) pixel on the edge, which is a special case, while still preserve the `roundOut` behavior for general non-edge cases.
Before the fix, notice a very thin gray line cutting through the purple box:
<img src="https://github.com/flutter/engine/assets/41930132/1482d81a-337e-4841-ac08-eff08bbc71ef" height="500">
Then after the fix, the gray line is gone:
<img src="https://github.com/flutter/engine/assets/41930132/0eddae69-ab62-4de6-8932-c67cc5aced73" height="500">
*List which issues are fixed by this PR. You must list at least one issue.*
https://github.com/flutter/flutter/issues/143420
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Address the performance of platform view due to an extra overlay. This overlay was added due to the following rounding problem:
> For example, if we interleave flutter widget and platform view in a list, and let's say we have a flutter widget (top = 0, bottom = 100.1), and a platform view below that widget (top = 100.1, bottom = 200). They are NOT supposed to be overlapping. However, after rounding out, we will get flutter widget (top = 0, bottom = 101), and platform view (top = 100, bottom 200). This will result in 1 pixel overlap as long as there's a floating point in the coord.
(Quote myself from the discussion below).
*List which issues are fixed by this PR. You must list at least one issue.*
Fixes https://github.com/flutter/flutter/issues/143420
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
### Motivation of the change:
Both dart and flutter are using fairly outdated gn-sdk without properly maintained. Currently @hjfreyer is working on version'ed IDK / SDK libs which requires changes in gn-sdk to use the right version of the libs in fuchsia/sdk/obj/{arch}-api-{level} rather than the one in the fuchsia/sdk/arch. But current implementation does not support choosing the right version.
### Blocking issue:
The new gn-sdk (in flutter/tools/fuchsia/gn-sdk) generates multiple BUILD.gn files rather than a large BUILD.gn the previous version created. So most of the build rules need to switch from the old `fidl:{api}` build rule to `fidl/{api}` rule. The same change will happen in the dart/sdk, i.e. http://go/dart-reviews/356924. But since the two repos cannot have one single atomic change, changing either side first will cause flutter to break. E.g. the linkage error caused by duplicated symbols will happen if we change the dart/sdk first, since in flutter, it will still refer to the old build rules in the middle.
### Solutions:
Ideally we can create redirect rules in the current `build/fuchsia` buildroot tree to redirect the old rules into the new one, so we can make the change in the flutter first then dart/sdk. But creating the rules is not trivial and will only be used once.
So an alternative solution is
- pause the dart/sdk -> flutter roll
- submit dart/sdk change (http://go/dart-reviews/356924)
- update this change to manually bring the dart/sdk change, namely the `dart_revision` in the DEPS file and signatures in the ci/licences.
- resume the dart/sdk -> flutter roll.
But it requires this change itself to be reviewed first, and I'd like to know your opinion before moving forward.
See corresponding dart/sdk change at http://go/dart-reviews/356924.
### //build/fuchsia/ from buildroot should be removed after this change.
Bug: [b/40935282](https://issues.chromium.org/issues/40935282?pli=1&authuser=0)
FYI: @hjfreyer
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
During some new development work that might make transform resets more expensive we realized that the resets were mostly coming from the calls to snap the transform to a pixel translate value and many of those were NOPs since the transform was already on a pixel translate value. This PR will avoid those trivially unnecessary reset operations.
This PR changes the format check on CI to use the command added in
https://github.com/flutter/engine/pull/50747.
Additionally, while making this change, I noticed that the CI check was
not checking the formatting of all files, and that as a result, files
were present in the repo with incorrect formatting. I have fixed the
formatting and fixed the check to always check all files.
This reverts commit 87171e73397f7edfd4c145fbaebdd7208954debf.
Internal performance test shows that this PR negatively affects startup time. I'll reland this PR in parts.
b/307872797
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This reverts commit d5d8b5de90f40644f23d309c0f68d03e299334eb.
The flutter/tests/skp_generator test (running in flutter_tester on
Linux) was failing, along with some customer tests.
In https://g-issues.skia.org/issues/305780908 Skia is removing the
default SkFontMgr. Previous work consolidated all references to
txt/platform.h and this replaces those last references. I attempted to
mirror the existing functionality, which still responds to GN flags and
the target platform.
After this PR, Flutter should not be depending on the default fontmgr
(and the defines in flutter_defines.gni) will maintain that behavior
until the legacy functions/methods are deleted from Skia. There were a
few tests that I missed on an earlier PR which relied on the default
font (helper added in #47493). These tests were failing because they
were making some assertions related to TextBlobs, which didn't work if
the (now-empty) Typeface they loaded had no glyphs. Thus, I added a few
extra asserts to make sure these textblobs *had* glyphs which make the
failing tests less mysterious, should this issue crop up again.
I cleaned up Flutter's BUILD.gn file for Skia a bit, deleting unused
targets related to the font managers. This involved fixing an implicit
dependency from //third_party/glfw/ to `Gdi32.lib` on Windows.
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
This PR adds multiview support for `ExternalViewEmbedder`.
## Nomenclature
The term **view** can be ambiguous in `ExternalViewEmbedder`, and therefore the following terms are used:
* A **native view** refers to the final drawing surface that to composite layers to. It is the "view" used in other places of the engine, such as `Shell::AddView`.
* A **platform view** refers a platform view, a layer that holds content to be embedded.
## Change
The lifecycle of `ExternalViewEmbedder` is changed:
<table>
<thead>
<tr>
<th>Before PR</th>
<th>After PR</th>
<th>How it's called</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan=2>BeginFrame</td>
<td>BeginFrame</td>
<td>Once per frame</td>
</tr>
<tr>
<td>PrepareFlutterView</td>
<td>Once per flutter view</td>
</tr>
<tr>
<td>SubmitFrame</td>
<td>SubmitFlutterView (renamed)</td>
<td>Once per flutter view</td>
</tr>
<tr>
<td>EndFrame</td>
<td>EndFrame (unchanged)</td>
<td>Once per frame</td>
</tr>
</tbody>
</table>
* `BeginFrame` should perform per-frame actions, such as merge-unmerging threads.
* `PrepareView` should perform per-native-view preparations, such as recording the view ID and view size.
This change is necessary because some actions in `PrepareView` needs to be refreshed at the beginning of drawing every native view.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The root slice of a surface frame was requesting the construction of an RTree from its DisplayList when it was built from a layer tree, but since the layer tree already does branch culling, it would not benefit from any further culling during `DisplayList::Dispatch`. Further, if there are platform views present in the layer tree then they will need an RTree to accurately convey "pixel ownership" information to the platform, but the root slice lives below any and all platform views, so it is the only slice that doesn't need an RTree for that case.
The cost of having an RTree in that slice was the accumulation of information and lists of rects that would never prove useful.
