If settings.merged_platform_ui_thread is set to kMergeAfterLaunch, then
the engine will be started on the UI thread. After engine setup
completes and the Dart isolate is loaded, the UI task runner will be
merged into the platform thread and all future Dart execution will run
on the platform thread.
This makes it possible for other work to run on the platform thread
while the engine starts.
See https://github.com/flutter/flutter/issues/163064
Changes required for this roll:
* addition of LLVM libc (which is now a dependency of libcxx)
* updates the to license script
* patching the verify_exported check to allow some symbols added by
libcxx
* annotations for clang-tidy warnings
fixes#112207
## What's new?
- Added a `view_id` on `UpdateSemantics`
- Added a `view_id` on `DispatchSemanticsAction`
- Piped the `view_id` all over creation
- Updated tests for these actions across the different platforms
- Added `FlutterEngineSendSemanticsAction` to the embedder API in order
to not break `FlutterEngineDispatchSemanticsAction`
- Using this view ID properly on the Windows platform (see
`engine/src/flutter/shell/platform/windows/flutter_windows_engine.cc`)
## How to test
1. Checkout
[foundation-plus-framework](https://github.com/canonical/flutter/pull/36)
from canonical/flutter
2. Merge this branch into it
3. Enable the "Narrator" screen reader on windows
4. Run the Multi window reference app (see
[PR](https://github.com/canonical/flutter/pull/36) for details)
5. Open up another window, and note that the right buttons and things
are being highlighted, as the screenreader would expect 🎉
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
Fixes https://github.com/flutter/flutter/issues/163429
If we synchronously call into dart ui when dispatching pointer events,
then we only end up scheduling about ~half as many frames as we should.
Nothing is blocked, no task starvation, et cetera. It should always be
safe to post pointer events as a new task instead.
Collection of changes to DlVertices::Builder and the stopwatch visualizer.
At a high level:
* improve performance of the stopwatch visualizer by pre-allocating storage (and sharing it across both visualizers), lookup up font once, and cache the debug frame rate used. Updates to use Dl types instead of SkTypes.
* Change DlVerticesBuilder to allow storing the bounds and use that in the visualizer, since we already know them. Make FML_CHECKS into dchecks, as the dart:ui vertices will already bounds check correctly - so these should only be necessary for debugging engine changes.
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
Unlike https://github.com/flutter/engine/pull/56738, does not touch the vsync API - which looks like it depends on scheduling behavior today, at least for iOS.
Reverts: flutter/engine#56738
Initiated by: jonahwilliams
Reason for reverting: speculative revert for framework failures.
Original PR Author: jonahwilliams
Reviewed By: {jason-simmons}
This change reverts the following previous change:
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
This also updates the vsync waiter to use RunNowOrPostTask, so that we start vsync events as early as possible.
Changes the following shell callbacks to flush the dart event loop:
* OnPlatformViewSetViewportMetrics
* OnPlatformViewDispatchPointerDataPacket
* OnPlatformViewDispatchPlatformMessage
* OnPlatformViewSetSemanticsEnabled
* OnPlatformViewSetAccessibilityFeatures
Using a new TaskRunner API RunNowAndFlushMessages. If the task runner can run tasks on the current thread, this will immediately invoke a callback and then post an empty task to the event loop to ensure dart listeners fire.
This also updates the vsync waiter to use RunNowOrPostTask, so that we start vsync events as early as possible.
Flutter_tester has a task observer that checks whether the test's Dart code has finished execution. If Dart no longer has live ports but does have pending microtasks, then flutter_tester should continue running and force a drain of the microtask queue.
Fixes https://github.com/flutter/flutter/issues/158129
Shell::OnPlatformViewDispatchPlatformMessage is called on the platform thread and must invoke the message handler on the UI thread. If the platform and UI threads are merged, then the handler can be run immediately. The shell can then post an empty task to the UI thread to make the UI message loop run the task observer that drains the microtask queue.
Fixes https://github.com/flutter/flutter/issues/156595
ui.Canvas and ui.SceneBuilder now use the DlPath object directly from the ui.Path object. This results in increased sharing of the wrapper objects which then increases the sharing of both the converted Impeller paths and Skia's volatile flag.
The VolatilePathTracker mechanism is deleted and rather than count the number of frames that a path is stable for, instead we count the number of times it is used for rendering. If a path is used 100 times in a single frame, it will become non-volatile and start being cached almost immediately. The cached Impeller paths are now also tracked for all instances of the same path, rather than for each call site that originated from a DisplayList dispatch.
This is a re-land of https://github.com/flutter/engine/pull/55006, except that we special case the navigation channel to immediately dispatch its message if the isolate is not yet running.
This preserves the existing behavior relied upon by several iOS add2app tests, as well as the still used embedder v1 - and potentially undicovered future embedders.
Reverts: flutter/engine#55006
Initiated by: jonahwilliams
Reason for reverting: still failling mac module test
Original PR Author: jonahwilliams
Reviewed By: {jason-simmons}
This change reverts the following previous change:
Reland of https://github.com/flutter/engine/pull/54975
Fixes the initial route behavior for iOS. Previously the initial route setting would _always_ be posted as a task, which after merging the threads would fire after isolate creation, thus it would not actually update the initial route setting. Fixed Engine constructor so that it reads the initial route from the settings.
Reland of https://github.com/flutter/engine/pull/54975
Fixes the initial route behavior for iOS. Previously the initial route setting would _always_ be posted as a task, which after merging the threads would fire after isolate creation, thus it would not actually update the initial route setting. Fixed Engine constructor so that it reads the initial route from the settings.
Reverts: flutter/engine#54975
Initiated by: jtmcdole
Reason for reverting: I believe this change caused flutter/flutter to break in an engine roll. It was one of two changes - the other being a webui change.
Tests broken:
module_test_ios: testDualCold
router test:
```
[2024-09-05 17:43:20.837343] [STDOUT] stderr: [ +135 ms] VMServiceFlutterDriver: Connected to Flutter application.
[2024-09-05 17:43:20.841927] [STDOUT] stdout: [ +4 ms] 00:00 [32m+0[0m:
Original PR Author: jonahwilliams
Reviewed By: {jason-simmons, johnmccutchan}
This change reverts the following previous change:
If we use runNowOrPostTask on platform channel responses, then we may not wake up the dart message loop. If nothing else wakes it up, then the embedder may hang on platform channel responses.
This fixes several google3 integration tests when run in merged thread mode.
If we use runNowOrPostTask on platform channel responses, then we may not wake up the dart message loop. If nothing else wakes it up, then the embedder may hang on platform channel responses.
This fixes several google3 integration tests when run in merged thread mode.
Experimentally support merging UI and platform thread on Android/iOS. This works by changing the shell holder to set the UI thread to the platform thread. Several shell APIs that post messages from the platform to ui thread were changed to use RunNowOrPostTask to immediately call the UI task if the threads are the same.
Experimentally, this seems to work reasonably well if there are no platform views. On Android with TLHC it works fine either way, while iOS currently takes a big performance hit.
This can be opted into via a Plist:
```
<key>FLTEnableMergedPlatformUIThread</key>
<true/>
```
Or via AndroidManifest.xml:
```
<meta-data
android:name="io.flutter.embedding.android.EnableMergedPlatformUIThread"
android:value="true" />
```
https://github.com/flutter/flutter/issues/150525
The size of the LTO build of the engine with the dylib compressed is as follows:
```sh
$ du -k libFlutter*
5236 libFlutter.dylib.tar.gz
4324 libFlutterSlimpeller.dylib.tar.gz
```
Sizes are in KiB. This represents a binary size reduction of 17.41% of the compressed artifacts. The compression ratios will likely differ based on the compression scheme.
Uncompressed, the sizes are:
```sh
$ du -k libFlutter*
16920 libFlutter.dylib
14044 libFlutterSlimpeller.dylib
```
This represents a binary size reduction of 16.99% which is in the same ballpark.
The really mucky bit was backing out the raster cache and persistent cache. I want to clean that up in a later patch so that those TUs are part of a separate submodule.
Opting out of Impeller will lead to a fatal log at startup saying the opt-out is disallowed.
Fixes https://github.com/flutter/flutter/issues/126606
Part of https://github.com/flutter/flutter/issues/144439
This does two things:
* Logs a warning when the embedder requests a non-Impeller preference when creating a shell.
* Makes the iOS embedder request a warning be logged when Impeller is not used.
I decided to put the warning logs in the shell so that as we get more opinionated about Impeller on other platforms, those platforms can just flip a flag with common log origin.
This is a prototype of the [PlatformIsolate
API](https://github.com/flutter/flutter/issues/136314).
**UPDATE (Jan 25):** The PR is ready for review. PTAL.
The `PlatformIsolate` creation flow is:
1. `PlatformIsolate.spawn` running on parent isolate
(platform_isolate.dart)
a. Create `isolateReadyPort`
b. `PlatformIsolateNativeApi::Spawn` (platform_isolate.cc)
c. `DartIsolate::CreatePlatformIsolate` (dart_isolate.cc)
d. Isolate created. Entry point invocation task dispatched to platform
thread
e. `PlatformIsolate.spawn` returns a `Future<Isolate>`
2. On the platform thread, `_platformIsolateMain` is invoked in the
platform isolate
a. Create `entryPointPort`
b. Send `Isolate.current` metadata and `entryPointPort` back to the
parent isolate via `isolateReadyPort`
3. Back in the parent isolate, `isolateReadyPort.handler` is invoked
a. Send the user's `entryPoint` and `message` to the platform isolate
via `entryPointPort`
b. Use received isolate metadata to create a new `Isolate` representing
the platform isolate and complete the `Future<Isolate>`
4. In the platform isolate, `entryPointPort.handler` is invoked
a. Run the user's `entryPoint(message)`
The engine shutdown flow is handled by `PlatformIsolateManager`, which
maintains a set of running platform isolates.
Fixes https://github.com/flutter/flutter/issues/141571
Shell::Screenshot was impelemnted in a Skia-only way and would crash when invoked. Adds test coverage for the breaking path on iOS that ends up calling `FlutterView drawLayer`.
This is part of the work towards supporting OpenGLES and Vulkan for runtime stage shaders.
Removes some redundant work we had around SkSL. Now only bundles the shaders we actually ask for from the command line.
@bdero, we should figure out if this is the right approach for flutter_gpu.
With this change, the IPLR format goes from having a root table of shader related information to a root table of shader information per `sksl`, `metal`, `opengles`, and `vulkan` platforms.
This may end up allowing us to revert https://github.com/flutter/engine/pull/47278, but I'm not sure I understand all the implications of that at this point.
I have run some but not all tests locally.
When we enter hybrid composition mode we 'pause' the default RenderSurface (implemented by SurfaceView or TextureView) and swap to an ImageReader based RenderSurface.
When we return from hybrid composition mode we recreate and re-initialize the real RenderSurface as if it was being used for the first time.
This broke Platform Views in an internal app b/306122497 because we would incorrectly tell the texture to attach when it was never detached.
This CL changes the protocol so that when we return from hybrid composition mode we only swap the RenderSurface and do not re-create it. This avoids doing a bunch of unnecessary work and fixes the logic error of re-attaching textures that were never detached.
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue.*
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
As far as I can tell, this functionality has _always_ been falling back to software rendering with Skia when Impeller was enabled. This is at best extremely misleading. Since I started putting Impeller specific objects in the display list it introduced crashes, and must be disabled.
Fixes https://github.com/flutter/flutter/issues/136847
This patch does the following:
- Updates `flutter_tester` to set up an Impeller rendering context and surface if `--enable-impeller` is set to true, using the Vulkan backend with Swiftshader.
- Updates `run_tests.py` to run all tests except the smoke test (that one really has no rendering impact whatsoever) with and without `--enable-impeller`.
- Updates a few tests to work that were trivial:
- A couple tests needed updated goldens for very minor rendering differences. Filed https://github.com/flutter/flutter/issues/135684 to track using Skia gold for this instead.
- Disabled SKP screenshotting if Impeller is enabled, and updated the test checking that to verify an error is thrown if an SKP is requested.
- The Dart GPU based test now asserts that the gpu context is available if Impeller is enabled, and does not deadlock if run in a single threaded mode.
- We were missing some trace events around `Canvas::SaveLayer` for Impeller as compared to Skia.
- A couple other tests had strict checks about exception messages that are slightly different between Skia and Impeller.
- I've filed bugs for other tests that may require a little more work, and skipped them for now. For FragmentProgram on Vulkan I reused an existing bug.
This is part of my attempt to address https://github.com/flutter/flutter/issues/135693, although @chinmaygarde and I had slightly different ideas about how to do this.
The goals here are:
- Run the Dart unit tests we already have with Impeller enabled.
- Enable running more of the framework tests (including gold tests) with Impeller enabled.
- Run all of these tests via public `dart:ui` API rather than mucking around in C++ internals in the engine.
This PR refactors `Rasterizer` so that it's more suitable for multi-view.
Design doc: [flutter.dev/go/multi-view-pipeline-and-rasterizer](http://flutter.dev/go/multi-view-pipeline-and-rasterizer)
With this change, `Rasterizer::DrawToSurfaces` has a structure that can handle drawing to multiple views, although the lack of some functionality still blocks it, mostly related to `ExternalViewEmbedder` and `RasterCache`.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This PR enforces the rules as documented in `FlutterView.Render`, where
calls in illegal situations should be ignored - but have never been
enforced.
```
/// This function must be called within the scope of the
/// [PlatformDispatcher.onBeginFrame] or [PlatformDispatcher.onDrawFrame]
/// callbacks being invoked.
///
/// If this function is called a second time during a single
/// [PlatformDispatcher.onBeginFrame]/[PlatformDispatcher.onDrawFrame]
/// callback sequence or called outside the scope of those callbacks, the call
/// will be ignored.
```
This rule is very important to implementing multi-view without having to
introduce new APIs. However, currently these illegal calls are not
ignored, and historically many tests (especially integration tests) were
unknowingly running based on this fact. @goderbauer did great work by
eliminating these cases in g3, and it's time for us to make sure these
calls are ignored.
Most effort of this PR goes to unit testing the changes. Some part of
`Shell::Create` is extracted into a static function to avoid duplicate
code.
## Pre-launch Checklist
- [ ] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [ ] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [ ] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [ ] I added new tests to check the change I am making or feature I am
adding, or Hixie said the PR is test-exempt. See [testing the engine]
for instructions on writing and running engine tests.
- [ ] I updated/added relevant documentation (doc comments with `///`).
- [ ] I signed the [CLA].
- [ ] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
