The Scene of the HTML renderer is passing incorrect size information to the engine, and when DPR<1, it can result in elements being culled off of the viewport.
In addition to that, when an app is embedded, not all changes in DPR cause a Resize event (only those some of the dimensions fails by a rounding error!), so this PR ensures that all DPR events in embedded trigger a resize event.
### Issues
Fixes https://github.com/flutter/flutter/issues/129182
### Testing
Looking good at: https://dit-astral-test.web.app
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The number of arguments are not used.
And also, even if we need it in the future, they can be derived at compile time:
```cpp
template <typename T>
struct function_traits;
template <typename Ret, typename... Args>
struct function_traits<Ret(Args...)>
{
using params = std::tuple<Args...>;
};
template <typename T>
constexpr std::size_t get_parameter_count() {
return std::tuple_size<typename function_traits<T>::params>::value;
}
template <typename T>
struct member_function_traits;
template <typename C, typename Ret, typename... Args>
struct member_function_traits<Ret(C::*)(Args...)>
{
using params = std::tuple<Args...>;
};
template <typename T>
constexpr std::size_t get_member_function_parameter_count() {
return std::tuple_size<typename member_function_traits<T>::params>::value;
}
```
(I got the code above with ChatGPT but I verified that they work)
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This PR adds a view ID parameter to the Dart FFI `_render` function, and
also remove the logic that skips illegal renders that violates the
render rule.
I decided to implement the change to add the view ID as a separate PR
because it's blocking us from performing benchmark testing internally,
which is required to verify before merging the full multiview pipeline.
We're also abolishing the render rule for now to allow presenting the
warmup frame, which affects the startup performance. It is planned to
implement the render rule once we can coordinate the warmup frame into
the pipeline.
### More on removing the render rule enforcement
The project needs to be divided into two phases:
* Phase 1: Dart:ui doesn't kip out-of-vsync frames. And the pipeline
presents these frames in a hacky way.
* Phase 2: Dart:ui skip out-of-vsync frames, but submit warmup frames
using a dedicated method. The pipeline contains no hacks.
The current logic only enforces the render rule in debug mode. It's not
desired to have the debug mode and the release mode behaving
differently. Moreover, I'd like to make only the necessary changes for
phase 1, so that there is as little "to be used in the future" code
between the two phases as possible.
## Pre-launch Checklist
- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I signed the [CLA].
- [ ] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
This fixes an issue when the same message was sent to the screen reader
again and was not subsequently announced by VoiceOver despite the prior
message having long been removed from the DOM.
Fixes https://github.com/flutter/flutter/issues/142250
Reland of: https://github.com/flutter/engine/pull/50139
Metal does not seem to like it when we collect 50+ command buffers at once. Adjust the aiks context logic to regularly flush the cmd buffers.
Reverts flutter/engine#50139
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
The Impeller Vulkan backend benefits from batching submission to the vk graphics queue. Managing this automatically is non-trivial and adds surprising/fragile thread based behavior, see: https://github.com/flutter/engine/pull/49870
Instead, introduce an impeller::CommandQueue object that command buffers must be submitted to in lieu of CommandBuffer->Submit, which has been made private.
TLDR
old
```c++
buffer->Submit();
```
new
```c++
context.GetQueue()->Submit({buffer});
```
The Metal and GLES implementations internally just call the private CommandBuffer->Submit, though there may be future opportunities to simplify here. The Vulkan implementation is where the meat is.
Aiks takes advantage of this by storing all command buffers on the aiks context while rendering a frame, and then performing one submit in aiks_context render. I don't think this will introduce any thread safety problems, as we don't guarantee much about aiks context - nor do we use it in a multithreaded context as far as I know.
Other tasks such as image upload still just directly submit their command buffers via the queue.
Fixes https://github.com/flutter/flutter/issues/141123
The Impeller Vulkan backend benefits from batching submission to the vk graphics queue. Managing this automatically is non-trivial and adds surprising/fragile thread based behavior, see: https://github.com/flutter/engine/pull/49870
Instead, introduce an impeller::CommandQueue object that command buffers must be submitted to in lieu of CommandBuffer->Submit, which has been made private.
TLDR
old
```c++
buffer->Submit();
```
new
```c++
context.GetQueue()->Submit({buffer});
```
The Metal and GLES implementations internally just call the private CommandBuffer->Submit, though there may be future opportunities to simplify here. The Vulkan implementation is where the meat is.
Aiks takes advantage of this by storing all command buffers on the aiks context while rendering a frame, and then performing one submit in aiks_context render. I don't think this will introduce any thread safety problems, as we don't guarantee much about aiks context - nor do we use it in a multithreaded context as far as I know.
Other tasks such as image upload still just directly submit their command buffers via the queue.
Fixes https://github.com/flutter/flutter/issues/141123
When a view gets disposed of, its rasterizer completely clears up the singleton `PlatformViewManager`. Particularly, it removes all registered platform view factories.
This is wrong because the remaining PlatformViews on the page cannot be re-rendered, and the default Platform View factories (used by `pointer_interceptor`, for example), disappear.
This PR attempts to preserve the `dispose` logic of the canvaskit rasterizer, without using the `debugClear` method of the `PlatformViewManager` (which is supposedly test-only).
## Issues
* Fixes https://github.com/flutter/flutter/issues/142094
## Tests
* Added unit-test
* Deployed demo app: https://dit-maps-tests.web.app
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Generated by https://github.com/flutter/engine/pull/48903 (`dart ./tools/header_guard_check/bin/main.dart --fix`).
As discussed with @cbracken and @jmagman, the guards are not technically needed on the Mac/iOS code, but they (a) do not hurt and (b) still provide value if for some reason `#include` is used instead of `#import` (though I suspect we could try to add that to the tool in the future as well).
Adds a Dart API so Application/Plugin programmers can retrieve the `initialData` configuration value that may be passed when adding a view from JS in a multiViewEnabled app.
When adding a view to an app like this:
```js
flutterApp.addView({
hostElement: someElement,
initialData: {
randomUUID: globalThis.crypto.randomUUID(),
}
});
```
`initialData` can be accessed from Dart by defining a JS-interop class like:
```dart
import 'dart:js_interop';
// The JS-interop definition of the `initialData` object passed to the views of this app.
@JS()
@staticInterop
class InitialData {}
/// The attributes of the [InitialData] object.
extension InitialDataExtension on InitialData {
external String? get randomUUID;
}
```
And then, from the code of the application:
```dart
...
Widget build(BuildContext context) {
final int viewId = View.of(context).viewId;
final InitialData? data = ui_web.views.getInitialData(viewId) as InitialData?;
return Text('${data?.randomUUID}');
}
...
```
## Testing
Will add unit tests once naming is sorted out :)
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
closesflutter/flutter#53107
This PR introduces support for `AppLifecycleState` on Web, aligning the web's lifecycle events with those of the mobile platforms. This ensures a more consistent developer experience and better lifecycle management across all platforms.
**PR includes:**
- Page Visibility Handling: Integrated the `visibilitychange` DOM event to determine if the app is in a `resumed` or `paused` state based on the visibility state of the document.
- Page Transition Handling: Used `beforeunload` events to better manage the `detached` state.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
The backend specific sampler libraries hold a strong reference to the native sampler objects and never clear this cache. As a result of this, we don't theoretically need rendering commands to increment a shared_ptr ref count - instead the sampler library can provide the Sampler object as a const ref and guarantee that it continues to be valid.
This allows us to reduce the amount of refcount ops for commands that use samplers.
Additionally, the sampler library uses nullptr as a sentinel for failing to construct a sampler object. Since sampler already has an isValid member that is checked - we can replace this with a specific invalid object subtype.
## Description
On Web, browsers can emit key events with a logical key sets to `Process` during compostion.
It is usually not a problem, but in some edge cases (for instance when the browser window lost focus and some keys events were missed), the Flutter web engine might synthesize an up event with a logical key value different that the one used for the down event and this will lead to an assertion message on the framework side.
## Related Issue
Fixes https://github.com/flutter/flutter/issues/126247.
## Tests
Adds 1 test.
The original change was reverted as it broke the GPU tracer reset logic. The previous logic assumed that the first started command buffer would also be the first submitted command buffer. With the direct encoding, this is no longer the case - so we shift the logic so that we reset query pools once at startup and then after the queries are finished being recorded.
This might actually be better in general since we should be doing less work in the frame workload.
Fix equality and hash code in `ParagraphStyle` and `StrutStyle` across renderers. Since those classes transitively rely on `Paint` and other classes, also fix those at least to a point that they pass the `test/painting/text_style_test.dart` test on the framework side.
Reverts flutter/engine#49780
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
Part of https://github.com/flutter/flutter/issues/140804
Rather than using impeller::Command, the impeller::RenderPass records most state directly into the Vulkan command buffer. This should remove allocation/free overhead of the intermediary structures and make further improvements to the backend even easier. This required a number of other changes to the renderer:
1. The render pass holds a strong ptr to the context. This helps avoid locking continually while encoding, which is quite slow.
2. barriers need to be encoded on the _producing_ side, and not the consuming side. This is because we'll actually run the consuming code before the producing code. i.e. we transition to shader read at the end of a render pass instead of when binding.
3. I've updated the binding code to also provide the descriptor type so that we don't need to look it up from the desc. set.
4. I added a test render pass class that records commands.
Very simple tweak to flutter.js that allows blob URLs to be used as entrypointUrls when loading flutter (this is needed by dartpad)
## Testing
I had to test this locally by spinning up a flutter app with something like:
```html
<script>
let initSrc = `let actualMain = document.createElement("script");
document.head.appendChild(actualMain);
actualMain.id = "injected_from_a_blob";
actualMain.src = "main.dart.js";`;
let blobInit = new Blob([...initSrc]);
let blobUrl = URL.createObjectURL(blobInit);
</script>
<script>
window.addEventListener('load', function(ev) {
// Download main.dart.js
_flutter.loader.loadEntrypoint({
entrypointUrl: blobUrl,
onEntrypointLoaded: function(engineInitializer) {
engineInitializer.autoStart();
}
});
});
</script>
```
## Issues
* Fixes https://github.com/flutter/flutter/issues/141329
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Part of https://github.com/flutter/flutter/issues/140804
Rather than using impeller::Command, the impeller::RenderPass records most state directly into the Vulkan command buffer. This should remove allocation/free overhead of the intermediary structures and make further improvements to the backend even easier. This required a number of other changes to the renderer:
1. The render pass holds a strong ptr to the context. This helps avoid locking continually while encoding, which is quite slow.
2. barriers need to be encoded on the _producing_ side, and not the consuming side. This is because we'll actually run the consuming code before the producing code. i.e. we transition to shader read at the end of a render pass instead of when binding.
3. I've updated the binding code to also provide the descriptor type so that we don't need to look it up from the desc. set.
4. I added a test render pass class that records commands.
The Flutter GPU tests were broken, and it turns out that I had never set them up to run on CI. This fixes the HostBuffer and gets the test suite running on CI.
Part of https://github.com/flutter/flutter/issues/140804
Migrate the rest of the commands in impeller to use the new API. Hide RenderPass::AddCommand. On subsequent changes I will be able to begin making some of these methods virtual so we can add more direct pass through. Though the vulkan backend will be blocked on changes to descriptor sets: https://github.com/flutter/engine/pull/49686
Now that the HostBuffer isn't implementing Buffer, there is really no reason to have a Buffer/DeviceBuffer split. make BufferViews use a DeviceBuffer, and remove the data ptr from buffer views (as we can always reach into the device buffer).
Makes some other cleanups too, since we always use a std::shared_ptr<DeviceBuffer>, then we can eliminated the shared_from_this call for AsBufferView by making this a static method.
Fixes https://github.com/flutter/flutter/issues/122823
Fixes https://github.com/flutter/flutter/issues/129659
Fixes https://github.com/flutter/flutter/issues/123741
This patch makes runtime stage/fragment program stuff work on Vulkan for Android.
It will need flutter/flutter#140976 for that to become a reality for flutter_tools users.
Compiling with relaxed Vulkan semantics still has an issue: shaders that use `sampler2D` with an explicitly set `location` on the `layout` will fail to compile with an error documented in https://github.com/flutter/flutter/issues/141219.
I think there might still be some issues with fragment programs on Vulkan, but this should at least be a good starting point and unblocks ink_sparkle.frag usage in the framework.
I've deleted some runtime_stage related code that would never get used - for example, enum related code that indicates we might support a bunch of data types that we do not and probably never will support in this API.
Reland of https://github.com/flutter/engine/pull/49505
---
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
Reverts flutter/engine#49505
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
This PR makes some major revisions to our flutter.js bootstrapper.
* Modularize flutter.js into multiple files to make it a little simpler to manage from source code. They are still bundled into a single .js file by esbuild.
* Added a `types.d.ts` file which contains declarations of the types of some of the objects used in the flutter.js API
* Deprecated the old `FlutterLoader.loadEntrypoint` API and added a new function simply called `FlutterLoader.load`, which has a few more capabilities:
- A build tool can inject a build config, that may describe multiple builds that `FlutterLoader.load` can attempt to use. It will use the first one that is compatible with the browser environment and the user's configuration.
- It can also load wasm flutter apps.
- It also pre-loads and instantiates CanvasKit (and Skwasm) as necessary depending on the build configuration.
- `FlutterLoader.load` also immediately takes a flutter configuration object. If an `onEntrypointLoaded` callback is not provided by the user, it just does the expected thing and initializes the engine and immediately starts the app, passing the configuration along as needed.
* `flutter.js` has the engine hash built into it now, which allows it to ascertain the correct CDN URLs for both CanvasKit and Skwasm.
Make the wrapped HostBuffer wrapper track/look up emplacements using a
fake byte offset.
This is a trick to keep Flutter GPU working after
https://github.com/flutter/engine/pull/49505 lands. I'll likely swing
around and change how `BufferView` works later on. We can simplify a lot
by making Flutter GPU `BufferView`s just take `DeviceBuffer` handles.
* Switch from legacy uniform semantics to uniform structs.
* Completely separate shader bundle from runtime stage.
* Packing multiple backends per shader.
* Pack struct and member fields into the shader bundle flatbuffer.
* Bind uniforms with correct metadata for GLES.
* Add uniform struct size and member offset reflection.
This patch will be important for Android/Vulkan, where we'll need to bundle GLES, Vulkan, and SkSL (at least for now).
I've refactored impellerc_main to break up some of the larger methods into more granular ones.
I've also changed the way that the `IPLR` output works so that we create as many compilers as runtime stages requested.
We still are in a weird situation where we create one "Default" compiler for the reflection data and depfile. I don't think that needs to be addressed in this patch, but we should do something about this. I've talked a little with @bdero and @jonahwilliams - it will probably involve something like making the generated reflection data a bit more flexible than it currently is so that it can more ergonomically capture different platform anomalies - we could do this either by having a compiler type that is explicitly for generating reflection data rather than using one arbitrary one and making the reflector try to be platform agnostic. I've filed a couple TODOs related to this and linked them in the patch.
Fixes https://github.com/flutter/flutter/issues/140817
Make TextStyle implementations consistent across web renderers and consistent with the mobile implementation. This will allow us to unskip `test/painting/text_style_test.dart` on the framework side.
Fixes https://github.com/flutter/flutter/issues/138371
When a new route pops up the expectation is that the screen reader focuses on something in the new route. Since routes typically result in DOM nodes being replaced, the current effect is that the screen reader simply unfocuses from the page, causing the user to have to refocus on back on the page and look for elements to interact with, which is a poor user experience. The current workaround is to use `autofocus`, but that doesn't scale as it's easy to forget, and if the route in question is maintained by a different person you may not even have enough control over it to set `autofocus` on anything. For example, this is the case with Flutter's default date picker. All you have is `showDatePicker` and there's no way to control the focus.
With this change the route (managed by the `Dialog` primary role) will check if a widget requested explicit focus (perhaps using `autofocus`), and if not, looks for the first descendant that a screen reader can focus on, and requests focus on it. The auto-focused element does not have to be literally focusable. For example, plain `Text` nodes do not have input focus (i.e. they are not `isFocusable`) but screen readers can still focus on them. If such an element is found, the web engine requests that the browser move focus to it programmatically (`element.focus()`), which causes the screen reader to move the a11y focus to it as well, but it sets `tabindex=-1` so the element is not focusable via keyboard or mouse.
This is part of the work towards supporting OpenGLES and Vulkan for runtime stage shaders.
Removes some redundant work we had around SkSL. Now only bundles the shaders we actually ask for from the command line.
@bdero, we should figure out if this is the right approach for flutter_gpu.
With this change, the IPLR format goes from having a root table of shader related information to a root table of shader information per `sksl`, `metal`, `opengles`, and `vulkan` platforms.
This may end up allowing us to revert https://github.com/flutter/engine/pull/47278, but I'm not sure I understand all the implications of that at this point.
I have run some but not all tests locally.
