Fixes https://github.com/flutter/flutter/issues/138371
When a new route pops up the expectation is that the screen reader focuses on something in the new route. Since routes typically result in DOM nodes being replaced, the current effect is that the screen reader simply unfocuses from the page, causing the user to have to refocus on back on the page and look for elements to interact with, which is a poor user experience. The current workaround is to use `autofocus`, but that doesn't scale as it's easy to forget, and if the route in question is maintained by a different person you may not even have enough control over it to set `autofocus` on anything. For example, this is the case with Flutter's default date picker. All you have is `showDatePicker` and there's no way to control the focus.
With this change the route (managed by the `Dialog` primary role) will check if a widget requested explicit focus (perhaps using `autofocus`), and if not, looks for the first descendant that a screen reader can focus on, and requests focus on it. The auto-focused element does not have to be literally focusable. For example, plain `Text` nodes do not have input focus (i.e. they are not `isFocusable`) but screen readers can still focus on them. If such an element is found, the web engine requests that the browser move focus to it programmatically (`element.focus()`), which causes the screen reader to move the a11y focus to it as well, but it sets `tabindex=-1` so the element is not focusable via keyboard or mouse.
This is part of the work towards supporting OpenGLES and Vulkan for runtime stage shaders.
Removes some redundant work we had around SkSL. Now only bundles the shaders we actually ask for from the command line.
@bdero, we should figure out if this is the right approach for flutter_gpu.
With this change, the IPLR format goes from having a root table of shader related information to a root table of shader information per `sksl`, `metal`, `opengles`, and `vulkan` platforms.
This may end up allowing us to revert https://github.com/flutter/engine/pull/47278, but I'm not sure I understand all the implications of that at this point.
I have run some but not all tests locally.
This reverts commit 87171e73397f7edfd4c145fbaebdd7208954debf.
Internal performance test shows that this PR negatively affects startup time. I'll reland this PR in parts.
b/307872797
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
* Add `--shader-bundle` mode to impellerc that takes a simple JSON spec and produces a single flatbuffer with a pack of named shaders.
* Added "single invocation" mode to the impellerc GN templates.
* Record vertex attribute reflection information.
* Light refactoring of the compiler frontend to make compiler invocations easier to follow.
Example shader bundle spec (json form of the yaml spec as shown in the [Flutter GPU](https://docs.google.com/document/d/1Sh1BAC5c_kkuMVreo7ymBzPoMzb7lamZRPsI7GBXv5M/edit?resourcekey=0-5w8u2V-LS41tCHeoE8bDTQ#heading=h.a3gmnzue7wgq) doc):
```json
{
"UnlitFragment": {
"type": "fragment",
"file": "shaders/flutter_gpu_unlit.frag"
},
"UnlitVertex": {
"type": "vertex",
"file": "shaders/flutter_gpu_unlit.vert"
},
"TextureFragment": {
"type": "fragment",
"file": "shaders/flutter_gpu_texture.frag"
},
"TextureVertex": {
"type": "vertex",
"file": "shaders/flutter_gpu_texture.vert"
}
}
```
Example impellerc invocation:
```bash
impellerc \
--include=~/projects/flutter/engine/src/flutter/impeller/compiler/shader_lib \
--runtime-stage-metal \
--sl=assets/TestLibrary.shaderbundle \
--shader-bundle='{"UnlitFragment": {"type": "fragment", "file": "shaders/flutter_gpu_unlit.frag"}, "UnlitVertex": {"type": "vertex", "file": "shaders/flutter_gpu_unlit.vert"}, "TextureFragment": {"type": "fragment", "file": "shaders/flutter_gpu_texture.frag"}, "TextureVertex": {"type": "vertex", "file": "shaders/flutter_gpu_texture.vert"}}'
```
Runtime usage:
```dart
/// Add a render pass encoder to the command buffer so that we can start
/// encoding commands.
final encoder = commandBuffer.createRenderPass(renderTarget);
/// Load a shader bundle asset.
final library =
gpu.ShaderLibrary.fromAsset('assets/TestLibrary.shaderbundle')!;
/// Create a RenderPipeline using shaders from the asset.
final vertex = library['TextureVertex']!;
final fragment = library['TextureFragment']!;
final pipeline = gpu.gpuContext.createRenderPipeline(vertex, fragment);
encoder.bindPipeline(pipeline);
```
https://github.com/flutter/engine/assets/919017/6f3e9a59-d180-4ba6-b14c-fa6d7056965c
Currently, in multi-view mode, when the user resizes the window, Flutter views don't respond to the resize. This is because `FlutterViewEmbedder` is the one responding to resize events. But `FlutterViewEmbedder` only works with the implicit view, so multi views didn't respond to resize events.
This PR moves the responsibility of responding to resize events from `FlutterViewEmbedder` to `EngineFlutterView`. Also, tests.
Part of https://github.com/flutter/flutter/issues/134443
This PR adds `String? identifier` to `SemanticsUpdateBuilder` (currently it's only available in the temproary `SemanticsUpdateBuilderNew` API.
This is mainly targeted at https://github.com/flutter/flutter/issues/17988
Steps:
part 1: [engine] add `SemanticsUpdateBuilderNew` https://github.com/flutter/engine/pull/47961
part 2: [flutter] use `SemanticsUpdateBuilderNew` https://github.com/flutter/flutter/pull/138331
**part 3: [engine] update `SemanticsUpdateBuilder` to be the same as `SemanticsUpdateBuilderNew`** <-- we are here
part 4: [flutter] use (now updated) `SemanticsUpdateBuilder` again.
part 5: [engine] remove `SemanticsBuilderNew`
- Move initialization of `PointerBinding`/`KeyboardBinding` out of `FlutterViewEmbedder`.
- `computeEventOffsetToTarget` properly handles events within a given view.
- `PointerBinding` operates on the given Flutter view (it still listens to some `domWindow` events for the implicit view).
- Stop using globals e.g. `ui.window`, `KeyboardBinding.instance`, `SafariPointerEventWorkaround.instance`, etc.
- `pointer_binding_test.dart` doesn't use globals either.
- `clickDebouncer` is now a static property on `PointerBinding`.
Fixes https://github.com/flutter/flutter/issues/137289
Places the binding data in a vector, since they key was only meaningful on metal but not used anywhere. I don't think that we need to specificially handle the case where our own contents bind the same contents multiple times, but interested to discuss if folks disagree.
Placing the vertex buffer on the binding object meant that we were actually paying 2x the size cost for it (one for vertex bindings, one for fragment bindings). By moving this object onto command itself, we reduce the size and avoid spliting up the command state in a weird way.
Also updates most of the contents to prefer moving the VertexBuffer.
This PR adds multiview support for `ExternalViewEmbedder`.
## Nomenclature
The term **view** can be ambiguous in `ExternalViewEmbedder`, and therefore the following terms are used:
* A **native view** refers to the final drawing surface that to composite layers to. It is the "view" used in other places of the engine, such as `Shell::AddView`.
* A **platform view** refers a platform view, a layer that holds content to be embedded.
## Change
The lifecycle of `ExternalViewEmbedder` is changed:
<table>
<thead>
<tr>
<th>Before PR</th>
<th>After PR</th>
<th>How it's called</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan=2>BeginFrame</td>
<td>BeginFrame</td>
<td>Once per frame</td>
</tr>
<tr>
<td>PrepareFlutterView</td>
<td>Once per flutter view</td>
</tr>
<tr>
<td>SubmitFrame</td>
<td>SubmitFlutterView (renamed)</td>
<td>Once per flutter view</td>
</tr>
<tr>
<td>EndFrame</td>
<td>EndFrame (unchanged)</td>
<td>Once per frame</td>
</tr>
</tbody>
</table>
* `BeginFrame` should perform per-frame actions, such as merge-unmerging threads.
* `PrepareView` should perform per-native-view preparations, such as recording the view ID and view size.
This change is necessary because some actions in `PrepareView` needs to be refreshed at the beginning of drawing every native view.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Accompanying framework PR: https://github.com/flutter/flutter/pull/138331
This PR implements support for exposing `SemanticsProperties.identifier` on Android as `resource-id`. Mainly targeted at https://github.com/flutter/flutter/issues/17988. Would also fix https://github.com/flutter/flutter/issues/issues/137735 but it was marked as duplicate. Anyway, there's a lot of context in that issue.
This PR requires changing the `SemanticsUpdateBuilder` interface (defined in engine) that framework depends on, so it requires introducing a temporary API ([see question I asked on Discord](https://discord.com/channels/608014603317936148/608018585025118217/1174845658033819729) to learn more about this approach).
Steps:
**part 1: [engine] add `SemanticsUpdateBuilderNew`** <-- we are here
part 2: [flutter] use `SemanticsUpdateBuilderNew`
part 3: [engine] update `SemanticsUpdateBuilder` to be the same as `SemanticsUpdateBuilderNew`*
part 4: [flutter] use (now updated) `SemanticsUpdateBuilder` again.
part 5: [engine] remove `SemanticsBuilderNew`
I'd like to do these changes first, and only then continue with [the proper framework PR](https://github.com/flutter/flutter/pull/138331).
*More specifically: update `SemanticsUpdateBuilder.updateNode()` to be the same as `SemanticsUpdateBuilderNew.updateNode()`. Number of arguments that function takes is the only change.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The ulimit logic has been failing for me for months now and felt still
ran fine. I think we don't need it any more.
Also add the `-i` option for incremental runs of `felt`. It causes felt
to start faster because it doesn't run `pub get`. In the future, we can
also run felt from the snapshot for even faster start-up.
Implement multi-view for semantics. Main changes:
- `EngineSemanticsOwner` singleton is split into two classes:
- `EngineSemantics` is a singleton that manages global properties of the
semantics tree, such as whether semantics is currently enabled, and what
gesture mode is used.
- `EngineSemanticsOwner` now supports creating multiple instances of the
class. `EngineFlutterView` now points a `EngineSemanticsOwner` that it
owns.
- Fix a number of issues with disposal logic. Now that views can come in
and out disposal has to be more robust than previously.
`EngineSemanticsOwner` acquired a `reset()` and `SemanticsObject` acquired
a `dispose()` that clean up the tree and individual nodes respectively.
- In particular, this fixes an issue in Skwasm mode where slight
differences in asynchrony caused the semantics test to fail because old
nodes continued firing events after they were removed from the document
due to improper disposal.
- No implicit view in mult-view mode.
- `window.devicePixelRatio` => `EngineFlutterDisplay.instance.devicePixelRatio`.
- `window.physicalSize` => `view.physicalSize`.
- Remove `LayerTree.frameSize`.
- `defaultRouteName` is set to `/` when there's no implicit view.
- All routing operations are noops in multi-view mode.
With these changes, I was able to run all examples in https://github.com/goderbauer/mvp without an implicit view.
The root slice of a surface frame was requesting the construction of an RTree from its DisplayList when it was built from a layer tree, but since the layer tree already does branch culling, it would not benefit from any further culling during `DisplayList::Dispatch`. Further, if there are platform views present in the layer tree then they will need an RTree to accurately convey "pixel ownership" information to the platform, but the root slice lives below any and all platform views, so it is the only slice that doesn't need an RTree for that case.
The cost of having an RTree in that slice was the accumulation of information and lists of rects that would never prove useful.
Towards https://github.com/flutter/flutter/issues/134501.
This PR makes the following changes to the public dart:ui API:
* It adds the `FlutterView.pysicalConstraints` property that describes max and min width and height for a view. The framework is allowed to size the `FlutterView` to any `Size` that meets these constraints.
* It adds an optional `size` argument to `FlutterView.render`. The framework provides the chosen `Size` that meets the aforementioned constraints to the `render` method. If the `FlutterView.pysicalConstraints` are tight (minHeight == maxHeight and minWidth == maxWidth) the argument is optional to remain backwards compatible. In all other cases, a `Size` must be provided.
* It adds a `ViewConstraints` class, which is basically the `dart:ui` version of `BoxConstraints` (This is similar to how we have `ViewPadding` in dart:ui to mirror `EdgeInsets` from the framework). It describes the constraints of a `FlutterView`, i.e. it powers the `FlutterView.pysicalConstraints` property.
This change does not wire anything up to the embedders. For now, `FlutterView.pysicalConstraints` just returns tight constraints for the embedder-provided size of the view (`FlutterView.physicalSize`) and the size provided to `FlutterView.render` is ignored (after it is checked that it meets the constrains).
This PR enables the framework to implement the new dynamic view sizing and embedders to separately expose the new functionality to their clients.
Presubmits will fail until https://github.com/flutter/flutter/pull/138565 is submitted to the framework.
**DO NOT SUBMIT until https://github.com/flutter/flutter/pull/138648 is ready.**
First triangle, in the framework! 🎉
Adds shader libraries, pipelines, command buffers, render passes, etc.
* Light pipelines/shader objects. No optimization yet, pipeline warming
to come.
* "Dynamic" command style. Don't re-send bindings if you don't need to.
Essentially: https://github.com/flutter/flutter/issues/133179
* No need to explicitly encode passes.
* Minimal descriptor usage.
* Nothing is async, except for the optional command buffer completion
callback.
It took a bunch of experimenting to get here, but I think things are
starting to look pretty neat. :)
Todo:
* Land the shader bundle format/remove the testing hacks & fixtures that
piggyback off of the runtime effect system.
* Add remaining calls for blend config, clearing bindings, etc.
* Inconsistent error handling patterns that need cleanup.
* Maybe: Surface exceptions for validation errors.
* Handle the texture usage bitmask more elegantly.
