This is an attempt to reland the overlay optimization for skwasm and fixing the golden diffs from the framework tests.
Original PR description:
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Reverts: flutter/engine#55402
Initiated by: chingjun
Reason for reverting: caused internal tests to fail.
See b/369740500 for more details.
Original PR Author: harryterkelsen
Reviewed By: {yjbanov}
This change reverts the following previous change:
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
In the Vulkan backend, the CommandPoolRecyclerVK stores command pools that can be reused while rendering a frame. For each frame, SurfaceContextVK::AcquireNextSurface calls CommandPoolRecyclerVK::Dispose to clear the cached pools.
However, the CommandPoolRecyclerVK's cache of pools is placed in thread-local storage. So command pools used on the IO thread will not be cleaned up if frame rendering calls Dispose on the raster thread.
This PR adds a Context API that the IO thread can call after using a command pool in order to ensure that cached resources are disposed.
Fixes https://github.com/flutter/flutter/issues/155519
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
In certain situations, semantics elements get assigned `pointer-events: none` when they aren't supposed to.
One such situation is when a text field has a [decoration error text](https://api.flutter.dev/flutter/material/InputDecoration/errorText.html). The semantics node become a container, and we always set `pointer-events: none` on container nodes.
This PR introduces an `acceptsPointerEvents` getter on `SemanticRole` and `SemanticBehavior` to control when `pointer-events` should be `all` or `none`.
Fixes https://github.com/flutter/flutter/issues/141975
This is placeholder stuff that I added before the rest of the API to prove that we can call the engine symbols.
Today this is totally redundant as Flutter GPU has a bunch of automated tests which exercise every FFI call.
Part of https://github.com/flutter/flutter/issues/150953.
Provide a way to get the required minimum uniform byte alignment when referencing uniform blocks in a device buffer. Allow the user to explicitly flush DeviceBuffers (necessary for devices without shared memory).
Reverts: flutter/engine#54949
Initiated by: eyebrowsoffire
Reason for reverting: Incorrect golden diffs on engine roll, see https://github.com/flutter/flutter/pull/155181
Original PR Author: eyebrowsoffire
Reviewed By: {harryterkelsen}
This change reverts the following previous change:
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
ui.Canvas and ui.SceneBuilder now use the DlPath object directly from the ui.Path object. This results in increased sharing of the wrapper objects which then increases the sharing of both the converted Impeller paths and Skia's volatile flag.
The VolatilePathTracker mechanism is deleted and rather than count the number of frames that a path is stable for, instead we count the number of times it is used for rendering. If a path is used 100 times in a single frame, it will become non-volatile and start being cached almost immediately. The cached Impeller paths are now also tracked for all instances of the same path, rather than for each call site that originated from a DisplayList dispatch.
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
This removes usages of the deprecated top-level `instantiate` and
`invoke` methods and replaces it by method calls to the embedder
objects.
See [0] for the change that introduce this embedder API.
[0] https://dart-review.googlesource.com/c/sdk/+/383242/6
Relands https://github.com/flutter/engine/pull/54917. The change is the same as before, except now the native resources for `SkwasmColorFilter` and `SkwasmImageFilter` classes are no longer GC'd. Instead, we use manually managed native handles (vended and scoped by `withRawColorFilter` and `withRawImageFilter`). The bug in the previous PR was that filter objects were disposed with the paint while the framework continued holding onto them. When GC kicked the finalization registry, it attempted to double-free the filters.
The Dart analyzer will soon be changed so that if the `default` clause
of a `switch` statement is determined to be unreachable by the
exhaustiveness checker, a new warning of type
`unreachable_switch_default` will be issued. This parallels the behavior
of the existing `unreachable_switch_case` warning, which is issued
whenever a `case` clause of a `switch` statement is determined to be
unreachable.
Before adding the new warning to the analyzer, code in the engine needs
to first be updated to eliminate these unreachable `default` clauses, so
that the warning won't cause builds to break.
For more information, see
https://github.com/dart-lang/sdk/issues/54575.
In [some cases][1], text editing utilities re-focus the `<input />` element during a blur event. This causes an unusual sequence of `focusin` and `focusout` events, leading to the engine sending unintended events.
Consider the following HTML code:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: focus was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
</script>
</body>
</html>
```
Clicking input1, then input2, then input3 produces the following console logs:
```
// Input1 is clicked
focusin: focus was gained by <input type value=â"1" id=â"input1">â
// Input2 is clicked
focusout: focus is leaving <input type value=â"1" id=â"input1">â and it will go to <input type value=â"2" id=â"input2">â
focusin: focus was gained by <input type value=â"2" id=â"input2">â
// Input3 is clicked
focusout: focus is leaving <input type value=â"2" id=â"input2">â and it will go to <input type value=â"3" id=â"input3">â
focusin: focus was gained by <input type value=â"3" id=â"input3">â
```
Now, let's add a blur handler that changes focus:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: focus was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
input2.addEventListener('blur', (ev) => {
input2.focus();
});
</script>
</body>
</html>
```
The log sequence changes and gives the wrong impression that no dom element has focus:
```
// Input1 is clicked
focusin: focus was gained by <input type value=â"1" id=â"input1">â
// Input2 is clicked
focusout: focus is leaving <input type value=â"1" id=â"input1">â and it will go to <input type value=â"2" id=â"input2">â
focusin: focus was gained by <input type value=â"2" id=â"input2">â
// Input3 is clicked, but the handler kicks in and instead of the following line being a focusout, it results in a focusin call first.
focusin: focus was gained by <input type value=â"2" id=â"input2">â
focusout: focus is leaving <input type value=â"2" id=â"input2">â and it will go to null
```
In addition to that, during `focusout` processing, `activeElement` typically points to `<body />`. However, if an element is focused during a `blur` event, `activeElement` points to that focused element. Although, undocumented it can be verified with:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('document.hasFocus()', document.hasFocus());
console.log('document.activeElement', document.activeElement);
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
input2.addEventListener('blur', (ev) => {
input2.focus();
});
</script>
</body>
</html>
```
We leverage these behaviors to ignore `focusout` events when the document has focus but `activeElement` is not `<body />`.
https://github.com/flutter/flutter/issues/153022
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Adds the `crossOrigin` property to the `<img>` tag used for decoding. This allows us to use cross-origin images in CanvasKit as if it were loaded from the same origin, as long as the CORS headers are properly set on the server hosting the image.
In the case where the image doesn't have proper CORS headers, this change causes an error to occur while attempting to decode, rather than later on in Skia when the image is converted into a texture. Hitting the cross-origin error later causes Skia to behave in undefined ways.
Progress towards https://github.com/flutter/flutter/issues/149843
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Do not eagerly create an `SkPaint` object that's strongly referenced by `CkPaint`. Instead, when a `Canvas.draw*` is called, create a temporary `SkPaint` object, pass it to Skia, then immediately delete it. This way there are no persistent `SkPaint` handles lurking in the system that transitively hold onto expensive native resources.
Addresses the `Paint` issue in https://github.com/flutter/flutter/issues/153678 in CanvasKit
Spot checking some benchmarks. Here's the effect of this PR on `draw_rect_variable_paint`. It's a bit of a stress test as it creates 300K distinct `Paint` objects to render 600 pictures (a typical ratio does not normally exceed ten paints to one picture). Even so, the effect of creating an extra `SkPaint` on every `draw*` command does not look significant. However, removing a dependency on the GC for 300K objects looks like a good trade-off.
## Before
Allocation stats:
```
Paint Created: 300000
Paint Deleted: 300000
Paint Leaked: 300000
Picture Created: 600
Picture Deleted: 599
Picture Leaked: 599
```
Performance stats:
```
windowRenderDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
| average: 4679.551020408163 μs
| outlier average: 5100 μs
| outlier/clean ratio: 1.0898481452084188x
| noise: 3.11%
sceneBuildDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
| average: 4689.765306122449 μs
| outlier average: 5100 μs
| outlier/clean ratio: 1.087474461321549x
| noise: 3.19%
drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 8447.474226804125 μs
| outlier average: 9332.666666666666 μs
| outlier/clean ratio: 1.1047878236850721x
| noise: 3.52%
```
## After
Allocation stats:
```
Picture Created: 600
Picture Deleted: 599
Picture Leaked: 599
```
Performance stats:
```
windowRenderDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 4780.40206185567 μs
| outlier average: 5133.666666666667 μs
| outlier/clean ratio: 1.0738985131877936x
| noise: 2.70%
sceneBuildDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 4787.6082474226805 μs
| outlier average: 5133.666666666667 μs
| outlier/clean ratio: 1.0722821085936345x
| noise: 2.72%
drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 8243.309278350516 μs
| outlier average: 9033.333333333334 μs
| outlier/clean ratio: 1.0958382159768851x
| noise: 2.60%
```
Reverts: flutter/engine#54737
Initiated by: chingjun
Reason for reverting: Breaking internal tests. See b/363125155
Original PR Author: gaaclarke
Reviewed By: {matanlurey, jonahwilliams}
This change reverts the following previous change:
[This PR](https://github.com/flutter/engine/pull/54415) was reverted because it requires a manual roll into the framework.
issue: https://github.com/flutter/flutter/issues/127855
integration test: https://github.com/flutter/engine/pull/54415
This does the preliminary work for implementing wide gamut colors in the Flutter framework. Here are the following changes: 1) colors now specify a colorspace with which they are to be interpreted 1) colors now store their components as floats to accommodate bit depths more than 8
The storage of this Color class is weird with float/int storage but that is a temporary solution to support a smooth transition. Here is the plan for landing this:
1) Land this PR
1) Wait for it to roll into the Framework
1) Land https://github.com/flutter/flutter/pull/153938 which will make CupertinoDynamicColor implement Color
1) Land another engine PR that rips out the int storage: https://github.com/flutter/engine/pull/54714
Here are follow up PRs:
1) https://github.com/flutter/engine/pull/54473 - changes DlColor so the wide gamut colors are rendered
1) https://github.com/flutter/engine/pull/54567 - Hooks up these changes to take advantage of wide DlColor
1) https://github.com/flutter/flutter/pull/153319 - the integration test for the framework repo
There are some things that have been left as follow up PRs since they are technically breaking:
1) The math on `lerp` hasn't been updated to take advantage of the higher bit depth
1) `operator==` hasn't been updated to take advantage of the higher bit depth
1) `hashCode` hasn't been updated to take advantage of the higher bit depth
1) `alphaBlend` hasn't been updated to take advantage of the higher bit depth
1) `toString` hasn't been updated to take advantage of the higher bit depth
[This PR](https://github.com/flutter/engine/pull/54415) was reverted because it requires a manual roll into the framework.
issue: https://github.com/flutter/flutter/issues/127855
integration test: https://github.com/flutter/engine/pull/54415
This does the preliminary work for implementing wide gamut colors in the Flutter framework. Here are the following changes: 1) colors now specify a colorspace with which they are to be interpreted 1) colors now store their components as floats to accommodate bit depths more than 8
The storage of this Color class is weird with float/int storage but that is a temporary solution to support a smooth transition. Here is the plan for landing this:
1) Land this PR
1) Wait for it to roll into the Framework
1) Land https://github.com/flutter/flutter/pull/153938 which will make CupertinoDynamicColor implement Color
1) Land another engine PR that rips out the int storage: https://github.com/flutter/engine/pull/54714
Here are follow up PRs:
1) https://github.com/flutter/engine/pull/54473 - changes DlColor so the wide gamut colors are rendered
1) https://github.com/flutter/engine/pull/54567 - Hooks up these changes to take advantage of wide DlColor
1) https://github.com/flutter/flutter/pull/153319 - the integration test for the framework repo
There are some things that have been left as follow up PRs since they are technically breaking:
1) The math on `lerp` hasn't been updated to take advantage of the higher bit depth
1) `operator==` hasn't been updated to take advantage of the higher bit depth
1) `hashCode` hasn't been updated to take advantage of the higher bit depth
1) `alphaBlend` hasn't been updated to take advantage of the higher bit depth
1) `toString` hasn't been updated to take advantage of the higher bit depth
With [1] dart2wasm started using the new Wasm built-in string import
`wasm:js-string`.
With [2] dart2wasm started exporting `compile` and `compileStreaming`
from the generated JS runtime to compile dart2wasm-generated modules
with the `wasm:js-string` built-in, polyfilling it when not available.
Update Wasm compilation code in Flutter to use `compileStreaming` from
the JS runtime, enabling fast `jwasm:js-string` module when available.
[1]: 44c2e17600
[2]: 4cd6096da4
