This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
Fixes https://github.com/flutter/flutter/issues/155833
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out. It also fixes a performance issue caused by making too many Canvases just to record the size of the picture elements in the scene.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This is an attempt to reland the overlay optimization for skwasm and fixing the golden diffs from the framework tests.
Original PR description:
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures and shader masks that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Reverts: flutter/engine#55402
Initiated by: chingjun
Reason for reverting: caused internal tests to fail.
See b/369740500 for more details.
Original PR Author: harryterkelsen
Reviewed By: {yjbanov}
This change reverts the following previous change:
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
This is a reland of https://github.com/flutter/engine/pull/54878 with a fix for scenes with pictures that are eventually entirely clipped out.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This enhances the overlay optimization by delaying combining pictures to get tighter bounds for the pictures that make up the scene, enabling more sophisticated optimization since we can determine if they intersect with platform views on a per-picture basis.
Fixes https://github.com/flutter/flutter/issues/149863
On a Macbook in Chrome in an example app with an infinite scrolling grid of platform views, this brings the ratio of dropped frames from 93% to 55% (roughly 4 fps to 30 fps).
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
In certain situations, semantics elements get assigned `pointer-events: none` when they aren't supposed to.
One such situation is when a text field has a [decoration error text](https://api.flutter.dev/flutter/material/InputDecoration/errorText.html). The semantics node become a container, and we always set `pointer-events: none` on container nodes.
This PR introduces an `acceptsPointerEvents` getter on `SemanticRole` and `SemanticBehavior` to control when `pointer-events` should be `all` or `none`.
Fixes https://github.com/flutter/flutter/issues/141975
Reverts: flutter/engine#54949
Initiated by: eyebrowsoffire
Reason for reverting: Incorrect golden diffs on engine roll, see https://github.com/flutter/flutter/pull/155181
Original PR Author: eyebrowsoffire
Reviewed By: {harryterkelsen}
This change reverts the following previous change:
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
This PR refactors the scene builder's logic in order to more aggressively merge flutter content and platform view content together. This essentially covers the case discussed in this flutter issue: https://github.com/flutter/flutter/issues/149863
This optimization ensures that each picture or platform view is applied to the lowest possible slice in the scene, which avoids the proliferation of redundant slices and overlays in the scene.
In [some cases][1], text editing utilities re-focus the `<input />` element during a blur event. This causes an unusual sequence of `focusin` and `focusout` events, leading to the engine sending unintended events.
Consider the following HTML code:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: focus was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
</script>
</body>
</html>
```
Clicking input1, then input2, then input3 produces the following console logs:
```
// Input1 is clicked
focusin: focus was gained by <input type value=â"1" id=â"input1">â
// Input2 is clicked
focusout: focus is leaving <input type value=â"1" id=â"input1">â and it will go to <input type value=â"2" id=â"input2">â
focusin: focus was gained by <input type value=â"2" id=â"input2">â
// Input3 is clicked
focusout: focus is leaving <input type value=â"2" id=â"input2">â and it will go to <input type value=â"3" id=â"input3">â
focusin: focus was gained by <input type value=â"3" id=â"input3">â
```
Now, let's add a blur handler that changes focus:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: focus was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
input2.addEventListener('blur', (ev) => {
input2.focus();
});
</script>
</body>
</html>
```
The log sequence changes and gives the wrong impression that no dom element has focus:
```
// Input1 is clicked
focusin: focus was gained by <input type value=â"1" id=â"input1">â
// Input2 is clicked
focusout: focus is leaving <input type value=â"1" id=â"input1">â and it will go to <input type value=â"2" id=â"input2">â
focusin: focus was gained by <input type value=â"2" id=â"input2">â
// Input3 is clicked, but the handler kicks in and instead of the following line being a focusout, it results in a focusin call first.
focusin: focus was gained by <input type value=â"2" id=â"input2">â
focusout: focus is leaving <input type value=â"2" id=â"input2">â and it will go to null
```
In addition to that, during `focusout` processing, `activeElement` typically points to `<body />`. However, if an element is focused during a `blur` event, `activeElement` points to that focused element. Although, undocumented it can be verified with:
```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
<div id="container">
<input type="" value="1" id="input1">
<input type="" value="2" id="input2">
<input type="" value="3" id="input3">
</div>
<script>
container.addEventListener('focusin', (ev) => {
console.log('focusin: was gained by', ev.target);
});
container.addEventListener('focusout', (ev) => {
console.log('document.hasFocus()', document.hasFocus());
console.log('document.activeElement', document.activeElement);
console.log('focusout: focus is leaving', ev.target, 'and it will go to', ev.relatedTarget);
});
input2.addEventListener('blur', (ev) => {
input2.focus();
});
</script>
</body>
</html>
```
We leverage these behaviors to ignore `focusout` events when the document has focus but `activeElement` is not `<body />`.
https://github.com/flutter/flutter/issues/153022
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Adds the `crossOrigin` property to the `<img>` tag used for decoding. This allows us to use cross-origin images in CanvasKit as if it were loaded from the same origin, as long as the CORS headers are properly set on the server hosting the image.
In the case where the image doesn't have proper CORS headers, this change causes an error to occur while attempting to decode, rather than later on in Skia when the image is converted into a texture. Hitting the cross-origin error later causes Skia to behave in undefined ways.
Progress towards https://github.com/flutter/flutter/issues/149843
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Do not eagerly create an `SkPaint` object that's strongly referenced by `CkPaint`. Instead, when a `Canvas.draw*` is called, create a temporary `SkPaint` object, pass it to Skia, then immediately delete it. This way there are no persistent `SkPaint` handles lurking in the system that transitively hold onto expensive native resources.
Addresses the `Paint` issue in https://github.com/flutter/flutter/issues/153678 in CanvasKit
Spot checking some benchmarks. Here's the effect of this PR on `draw_rect_variable_paint`. It's a bit of a stress test as it creates 300K distinct `Paint` objects to render 600 pictures (a typical ratio does not normally exceed ten paints to one picture). Even so, the effect of creating an extra `SkPaint` on every `draw*` command does not look significant. However, removing a dependency on the GC for 300K objects looks like a good trade-off.
## Before
Allocation stats:
```
Paint Created: 300000
Paint Deleted: 300000
Paint Leaked: 300000
Picture Created: 600
Picture Deleted: 599
Picture Leaked: 599
```
Performance stats:
```
windowRenderDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
| average: 4679.551020408163 μs
| outlier average: 5100 μs
| outlier/clean ratio: 1.0898481452084188x
| noise: 3.11%
sceneBuildDuration: (samples: 98 clean/2 outliers/100 measured/300 total)
| average: 4689.765306122449 μs
| outlier average: 5100 μs
| outlier/clean ratio: 1.087474461321549x
| noise: 3.19%
drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 8447.474226804125 μs
| outlier average: 9332.666666666666 μs
| outlier/clean ratio: 1.1047878236850721x
| noise: 3.52%
```
## After
Allocation stats:
```
Picture Created: 600
Picture Deleted: 599
Picture Leaked: 599
```
Performance stats:
```
windowRenderDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 4780.40206185567 μs
| outlier average: 5133.666666666667 μs
| outlier/clean ratio: 1.0738985131877936x
| noise: 2.70%
sceneBuildDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 4787.6082474226805 μs
| outlier average: 5133.666666666667 μs
| outlier/clean ratio: 1.0722821085936345x
| noise: 2.72%
drawFrameDuration: (samples: 97 clean/3 outliers/100 measured/300 total)
| average: 8243.309278350516 μs
| outlier average: 9033.333333333334 μs
| outlier/clean ratio: 1.0958382159768851x
| noise: 2.60%
```
Previously the "dialog" ARIA role was only set if the route or any of its descendants had the `namesRoute` flag. This PR sets the ARIA role unconditionally. Although it's always a good idea to have a label on the route, a missing label doesn't change the fact that the route is a route, so the ARIA role should still be set.
Fixes https://github.com/flutter/flutter/issues/153791
Remember the timestamp of _all_ `pointerup` events, not just those that were flushed. Clicks should be deduplicated after a `pointerup` even when not debouncing anything. This is because when not debouncing the engine already forwards all the pointer events to the framework, and sending click events on top only causes double-clicks.
Fixes https://github.com/flutter/flutter/issues/147050
The `type` attribute is needed for screen readers to tell the user whether the text field is a password or a plain text field.
Use `-webkit-text-security` for multi-line fields as a best effort (it's non-standard but is supported by all major browsers).
Fixes https://github.com/flutter/flutter/issues/153801
This previously was basically assuming that two clip regions would be a union but instead it's actually an intersection. Also, for some reason I had forgotten to actually implement `boundingRect` for the rrect clip region. I changed this for a bit more clarity.
Enhances the overlay optimization by pushing new pictures to the earliest RenderCanvas they can go into. This improvement is made especially clear in the new test case I added to `embedded_views_test.dart`, with the previous algorithm we would have used an overlay for each platform view even though all of the pictures could go into the base canvas with no issue.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Rename "dialog" to "route" to match the framework. The ARIA role "dialog" is an implementation detail, and it might change if a better implementation comes along. The semantic node itself implements the framework's "route" concept.
A few non-functional clean-ups in semantics:
* Rename `RoleManager` to `SemanticBehavior`.
* Rename `PrimaryRoleManager` to `SemanticRole`.
* Remove the `Role` enum. Move the enum docs into the respective classes.
## Why?
Previous naming was confusing. It's not clear what the difference is between a "role manager" and a "primary role manager". The word "manager" is a meaningless addition; the `Semantic*` prefix is much more meaningful. The `Role` enum was only used for tests, but tests can just use `SemanticRole.runtimeType`.
## New state of the world
After this PR the semantics system has "objects" (class `SemanticsObject`), "roles" (class `SemanticRole`), and "behaviors" (class `SemanticBehavior`).
- A semantic _object_ is an object attached to the framework-side `SemanticNode`. It lives as long as the semantic node does, and provides basic functionality that's common across all nodes.
- A semantic object has exactly one semantic _role_. This role is determined from the flags set on the semantic node. Flags can change, causing a semantic object to change its role, which is why these are two separate classes. If an object had just one permanent role, we could combine these classes into one (maybe one day we'll do it, as changing roles dynamically is weird, but that needs major changes in the framework).
- A semantic role may have zero or more semantic _behaviors_. A behavior supplies a piece of functionality, such as focusability, clickability/tappability, live regions, etc. A behavior can be shared by multiple roles. For example, both `Button` and `Checkable` roles use the `Tappable` behavior. This is why there's a many-to-many relationship between roles and behaviors.
Or in entity relationship terms:
```mermaid
---
title: Semantic object relationships
---
erDiagram
SemanticsNode ||--|| SemanticsObject : managed-by
SemanticsObject ||--o{ SemanticRole : has-a
SemanticRole }o--o{ SemanticBehavior : has
```
The occlusion rectangle for platform views was going through this `inverseMapRect` code path, which actually was giving us the wrong results. The operations should just be doing the normal transformation on the rectangles to get the right result. It actually turns out we don't need the inverse mapping function, so I removed it, and I renamed the somewhat confusingly named `cullRect` function to `mapRect` which I think makes a bit more sense.
This should resolve https://github.com/flutter/flutter/issues/152139
This is in preparation for
https://dart-review.googlesource.com/c/sdk/+/373884, which will add length to the extension type itself. This will shadow the extension member, so we should make sure the type signatures match.
Reverts: flutter/engine#54002
Initiated by: kevmoo
Reason for reverting: the issue in google 3 has been fixed.
Original PR Author: auto-submit[bot]
Reviewed By: {fluttergithubbot}
This change reverts the following previous change:
Reverts: flutter/engine#54000
Initiated by: chingjun
Reason for reverting: Broke Google Testing.
See b/352191023 for more details.
Original PR Author: kevmoo
Reviewed By: {jonahwilliams}
This change reverts the following previous change:
Fixes https://github.com/flutter/flutter/issues/151679
Reverts: flutter/engine#54000
Initiated by: chingjun
Reason for reverting: Broke Google Testing.
See b/352191023 for more details.
Original PR Author: kevmoo
Reviewed By: {jonahwilliams}
This change reverts the following previous change:
Fixes https://github.com/flutter/flutter/issues/151679
Our treatment of the interaction between offset and transform was incorrect. Modified our platform view unit tests to cover more cases of nested offsets and transforms.
Always mark scrollable containers with `role="group"` to prevent the browser from merging all child elements into one giant string. Default to `preventScroll` to true across the web engine code, because it's a better default for Flutter than otherwise.
Fixes https://github.com/flutter/flutter/issues/130950
This relands https://github.com/flutter/engine/pull/53679.
The difference from the previous attempt is in the last commit, which prevents synthetic focus requests from echoing back into the framework. That part broke too many tests in g3 and needs to be revisited.
## Original description
Stop using `SemanticsAction.didGain/LoseAccessibilityFocus` on the web, start using `SemanticsAction.focus`. This is because on the web, a11y focus is not observable, only input focus is. Sending `SemanticsAction.focus` will guarantee that the framework move focus to the respective widget. There currently is no "unfocus" signal, because it seems to be already covered: either another widget gains focus, or an HTML DOM element outside the Flutter view does, both of which have their respective signals already.
More details in the discussion in the issue https://github.com/flutter/flutter/issues/83809.
Fixes https://github.com/flutter/flutter/issues/83809
Fixes https://github.com/flutter/flutter/issues/148285
Fixes https://github.com/flutter/flutter/issues/143337
Previously, each platform view contained styling/placement information from the entire stack of the layer builder. This caused issues when using `addRetained`, since it would contain stale styling/placement information from its old parent layers. I have changed it so that platform views only contain local styling information, and that styling is combined with the parent only when the layers are merged.
Second attempt to reland https://github.com/flutter/engine/pull/52023
Fixes since the previous reland attempt:
* We need to pass the skwasm main JS URI when loading the module so that it can pass that along to the worker. Since the worker uses the workaround to allow a cross script worker, it has trouble locating the main JS URI in relation to itself in a way that actually works for dynamic imports, so passing it along fixes that issue.
* Some of the Google3 tests relied on the relative default canvaskit path. Dynamic module imports seems to not handle relative paths the way we expect, so we do our own URL resolution using the URL constructor before passing it into the dynamic import API. Also cleaned up some of the other relative pathing stuff that we do around the base URI. in flutter.js
Reverts: flutter/engine#53783
Initiated by: jiahaog
Reason for reverting: There are still internal users of these hash functions (b/352191023). Flutter also exports it [here](72f83d3237/packages/flutter/lib/src/painting/basic_types.dart (L58)) so its easy for developers to use them even without importing `dart:ui`. Though it's an easy fix, I'd imagine this to be a breaking change for the OSS ecosystem a
Original PR Author: kevmoo
Reviewed By: {yjbanov}
This change reverts the following previous change:
We moved to dart:core Object functions a while ago
