Attempt to speed up vulkan startup by ensuring vulkan driver
initialization happens on raster thread. Prev. we immediately
initialized the impeller::Context, unfortunately setting up the vulkan
context can take upwards of 100ms. This time is running on the platform
thread and blocking startup.
Instead, I attempt to hide/defer as much as possible what backend is
being used - this requires us to access the impeller context via a
shared_future in some cases instead of immediately knowing it is valid.
<!-- start_original_pr_link -->
Reverts: flutter/flutter#167527
<!-- end_original_pr_link -->
<!-- start_initiating_author -->
Initiated by: gmackall
<!-- end_initiating_author -->
<!-- start_revert_reason -->
Reason for reverting: failing postsubmit `Linux_mokey
flutter_engine_group_performance`
<!-- end_revert_reason -->
<!-- start_original_pr_author -->
Original PR Author: jonahwilliams
<!-- end_original_pr_author -->
<!-- start_reviewers -->
Reviewed By: {jason-simmons}
<!-- end_reviewers -->
<!-- start_revert_body -->
This change reverts the following previous change:
Attempt to speed up vulkan startup by ensuring vulkan driver
initialization happens on raster thread. Prev. we immediately
initialized the impeller::Context, unfortunately setting up the vulkan
context can take upwards of 100ms. This time is running on the platform
thread and blocking startup.
Instead, I attempt to hide/defer as much as possible what backend is
being used - this requires us to access the impeller context via a
shared_future in some cases instead of immediately knowing it is valid.
<!-- end_revert_body -->
Co-authored-by: auto-submit[bot] <flutter-engprod-team@google.com>
Attempt to speed up vulkan startup by ensuring vulkan driver
initialization happens on raster thread. Prev. we immediately
initialized the impeller::Context, unfortunately setting up the vulkan
context can take upwards of 100ms. This time is running on the platform
thread and blocking startup.
Instead, I attempt to hide/defer as much as possible what backend is
being used - this requires us to access the impeller context via a
shared_future in some cases instead of immediately knowing it is valid.
This includes a fix for a race seen in
EmbedderTest.PlatformThreadIsolatesWithCustomPlatformTaskRunner
The implementaion of MergedPlatformUIThread::kMergeAfterLaunch required
changing the interface of the TaskObserverAdd/TaskObserverRemove
callbacks so that TaskObserverAdd returned the TaskQueueId where the
observer was added. That TaskQueueId would later be given to
TaskObserverRemove.
The original implementation of this PR updated the embedder library's
implementation of TaskObserverAdd to return TaskQueueId::kInvalid to
signal that the observer was not added. However, this conflicted with
the embedder's EmbedderTaskRunner, whose implementation of
GetTaskQueueId returns TaskQueueId::kInvalid as a placeholder.
This PR reverts the embedder's TaskObserverAdd/TaskObserverRemove to the
original implementation which adds the observer to the current thread's
message loop and does not call GetTaskQueueId.
See https://github.com/flutter/flutter/issues/167418
If settings.merged_platform_ui_thread is set to kMergeAfterLaunch, then
the engine will be started on the UI thread. After engine setup
completes and the Dart isolate is loaded, the UI task runner will be
merged into the platform thread and all future Dart execution will run
on the platform thread.
This makes it possible for other work to run on the platform thread
while the engine starts.
See https://github.com/flutter/flutter/issues/163064
Changes required for this roll:
* addition of LLVM libc (which is now a dependency of libcxx)
* updates the to license script
* patching the verify_exported check to allow some symbols added by
libcxx
* annotations for clang-tidy warnings
fixes#112207
## What's new?
- Added a `view_id` on `UpdateSemantics`
- Added a `view_id` on `DispatchSemanticsAction`
- Piped the `view_id` all over creation
- Updated tests for these actions across the different platforms
- Added `FlutterEngineSendSemanticsAction` to the embedder API in order
to not break `FlutterEngineDispatchSemanticsAction`
- Using this view ID properly on the Windows platform (see
`engine/src/flutter/shell/platform/windows/flutter_windows_engine.cc`)
## How to test
1. Checkout
[foundation-plus-framework](https://github.com/canonical/flutter/pull/36)
from canonical/flutter
2. Merge this branch into it
3. Enable the "Narrator" screen reader on windows
4. Run the Multi window reference app (see
[PR](https://github.com/canonical/flutter/pull/36) for details)
5. Open up another window, and note that the right buttons and things
are being highlighted, as the screenreader would expect 🎉
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
The isolate ownership API was [introduced
recently](https://dart-review.googlesource.com/c/sdk/+/407700) to solve
[some deadlock bugs](https://github.com/dart-lang/native/issues/1908) in
native callbacks.
A native callback is a call from native code into a Dart function.
Currently all such callbacks must run that Dart function in the isolate
that created the callback (called the target isolate). The only native
callback primitives at the moment are `NativeCallable.isolateLocal`
(blocking, but must be invoked from the same thread as the target
isolate, and the target isolate must be currently entered on that
thread) and `NativeCallable.listener` (non-blocking, can be invoked from
any thread).
To build blocking callbacks that can be called from any thread, we can
use a `NativeCallable.listener`, and use a synchronization object like a
mutex or a condition variable to block until the callback is complete.
However, if we try to do this on the thread that is currently entered in
the target isolate, we will deadlock: we invoke the listener, a message
is sent to the target isolate, and we block waiting for the message to
be handled, so we never pass control flow back to the isolate to handle
the message, and never stop waiting.
To fix this deadlock, Ffigen and Jnigen both have a mechanism that
checks if we're on the target isolate's thread first:
- If the native caller is already on the same thread as the target
isolate, and the target isolate is entered:
- Call the Dart function directly using `NativeCallable.isolateLocal` or
similar
- Otherwise, if the native caller is coming from a different thread:
- Call the Dart function asynchronously using `NativeCallable.listener`
or similar
- Block until the callback finishes
However, this neglects the case where we're on the target isolate's
thread, but not entered into the isolate. This case happens in Flutter
when the callback is invoked from the UI thread (or the platform thread
when thread merging is enabled), and the target isolate is the root
isolate. When the native callback is invoked, the root isolate is not
entered, so we hit the second case: we send a message to the root
isolate, and block to wait for a response. Since the root isolate is
exclusively run on the UI thread, and we're blocking the UI thread, the
message will never be handled, and we deadlock.
The isolate ownership API fixes this by allowing the embedder to inform
the VM that it will run a particular isolate exclusively on a particular
thread, using `Dart_SetCurrentThreadOwnsIsolate`. Other native code can
then query that ownership using `Dart_GetCurrentThreadOwnsIsolate`. This
lets us add a third case to our conditional:
- If the native caller is on the thread that is currently entered in the
target isolate:
- Call the Dart function directly using `NativeCallable.isolateLocal` or
similar
- Otherwise, if the native caller is on the thread that owns the target
isolate
- Enter the target isolate
- Call the Dart function directly using `NativeCallable.isolateLocal `or
similar
- Exit the target isolate
- Otherwise, the native caller is coming from an unrelated thread:
- Call the Dart function asynchronously using `NativeCallable.listener`
or similar
- Block until the callback finishes
**Note:** We don't need to set the ownership of VM managed threads,
because they run in a thread pool exclusively used by the VM, so there's
no way for native code to be executed on the thread (except by FFI, in
which case we're entered into the isolate anyway). We only need this for
Flutter's root isolate because work can be sent to the UI
thread/platform thread using OS specific APIs like Android's
`Looper.getMainLooper()`.
This PR introduces a `NativeAssetsManifest.json` next to the `AssetManifest.json` and `FontManifest.json`. This removes the need for embedding the native assets mapping inside the kernel file and will enable decoupling native assets building and bundling from the kernel compilation in flutter tools. This will then allow us to remove dry-run from the build hook protocol.
(It also means all isolate groups will have the same native assets. However, since Flutter does not support `Isolate.spawnUri` from kernel files anyways, this is not a regression.)
This manifest is parsed eagerly on startup by the engine in a manner similar to how the font manifest is parsed. The manifest contents need to be available in the callback for resolving assets, which does not have access to the engine. Therefore the parsed manifest is `NativeAssetsManager` stored in the `IsolateGroupData`. The engine passes it in on isolate group creation, and the FFI callbacks access it from the isolate group data.
Issue:
* https://github.com/flutter/flutter/issues/154425
Related PRs:
* https://dart-review.googlesource.com/c/sdk/+/388161
Follow up work:
* This PR does not yet remove the engine callbacks registered via the dart_api that rely on kernel embedding. If we were to do that in this PR, it would require a manual roll of the engine into flutter/flutter with the PR that switches flutter_tools to emit the native assets manifest instead of embedding in kernel, and a manual roll into g3 to switch emitting a manifest instead of embedding in kernel. A TODO is left in the code for those callbacks to be removed.
## Testing
Most of this PR cannot be tested in isolation. The code in this PR is heavily exercised in the follow up flutter_tools PR which creates the `NativeAssetsManifest.json` and removes the embedding of native assets in kernel files.
* This PR adds a unit test for parsing the JSON manifest.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Flutter_tester has a task observer that checks whether the test's Dart code has finished execution. If Dart no longer has live ports but does have pending microtasks, then flutter_tester should continue running and force a drain of the microtask queue.
Fixes https://github.com/flutter/flutter/issues/158129
When running with merged platform and ui threads, set the dart thread name of the main thread to io.futter.ui. Also change the thread mask settings to avoid creating an unused UI thread.
ui.Canvas and ui.SceneBuilder now use the DlPath object directly from the ui.Path object. This results in increased sharing of the wrapper objects which then increases the sharing of both the converted Impeller paths and Skia's volatile flag.
The VolatilePathTracker mechanism is deleted and rather than count the number of frames that a path is stable for, instead we count the number of times it is used for rendering. If a path is used 100 times in a single frame, it will become non-volatile and start being cached almost immediately. The cached Impeller paths are now also tracked for all instances of the same path, rather than for each call site that originated from a DisplayList dispatch.
This is a re-land of https://github.com/flutter/engine/pull/55006, except that we special case the navigation channel to immediately dispatch its message if the isolate is not yet running.
This preserves the existing behavior relied upon by several iOS add2app tests, as well as the still used embedder v1 - and potentially undicovered future embedders.
We used to require this only on iOS because the standard library till iOS 9 didn't have support for this. We have moved past that version. No change on other platforms.
This initializes the `NativeAssetsApi` for native assets resolution in the isolate group creation callback.
* https://github.com/dart-lang/sdk/issues/55523
## Implementation considerations
The DartIO initialization lives in its own GN target. This doesn't work for the native assets initialization due to it having to look up the `script_uri` in one of the callbacks. Since the callbacks are function pointers, we can't have a lambda that captures the script uri. So instead, the native assets initialization lives in the flutter/runtime target.
The import from dart should probably be `runtime/include/bin/native_assets_api.h` to mirror what we're doing with dart IO, rather than directly importing from `runtime/bin/native_assets.h`.
## Testing
All native asset testing is in flutter_tools, so those tests will only run once this rolls into flutter/flutter. I have done manual testing locally with a Dart branch that removes the fallback: https://dart-review.googlesource.com/c/sdk/+/370740
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This is a refactor that moves the `PointerDataPacketConverter` from `PlatformView` to `RuntimeController`.
This change is made for the following reasons:
- Currently, the pointer data conversion contains no platform specific logic (because the current converter's only responsibility is to make the event sequence conform Flutter's protocol). Therefore these logics should reside in a platform-independent place.
- The converter typically converts one event to many. It's better to have this conversion later than earlier.
- It removes a member from `PlatformView`, making it closer to a pure virtual class.
The reason to choose `RuntimeController` as the destination is because `RuntimeController` manages a map for views, which is required for the converter to implement a later patch https://github.com/flutter/engine/pull/51925.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Some embedders use a custom platform task runner that does not support fml::MessageLoopTaskQueues and may not have an fml::MessageLoop.
To support those embedders, DartIsolate::SetMessageHandlingTaskRunner will not use fml::MessageLoopTaskQueues for platform isolates.
See https://github.com/flutter/engine/pull/48551#issuecomment-1962190896
