This removes skips for the golden tests in `//testing/dart/canvas_test.dart` and instead passes them up to Skia gold.
Adds a utility class for dealing with Skia gold from these tests, as well as the existing fuzzy identical image comparison for tests that just want to do in memory comparison of images generated from the same test.
Removes the old golden files that were in tree.
Part of https://github.com/flutter/flutter/issues/53784
As part of eliminating the Flutter buildroot (https://github.com/flutter/flutter/issues/67373), we are moving all third-party dependencies from //third_party to //flutter/third_party.
Once all third-party dependencies have been migrated, tooling and config will be moved and the buildroot will be eliminated altogether.
No tests changed because there is no semantic change to this PR. This is simply relocating a dependency.
Adds a gtest test fixture mixin and convenience class that instantiates an NSAutoreleasePool at the beginning of each test and flushes it at the end; this allows Objective-C tests using ARC to free allocations at the end of each test.
Adds a subclass for the macOS accessibility bridge tests that instantiates and re-uses an NSWindow* across any tests that use it. This is because instantiating, closing, and immediately collecting an NSWindow results in a crash.
Prior to this patch, tests started failing (on my machine) around the 855th iteration, and issued the following warning on the 101st iteration:
```
2023-10-26 13:02:45.390829-0700 flutter_desktop_darwin_unittests[40837:1509026] [Window] WARNING: NSWindow has detected an excessive live window count of 101. Window 0x1423 of class 'NSWindow' created after passing the threshold of 100. This window is not necessarily the cause, and this warning will only be shown once per window class. (
0 AppKit 0x0000000192820d28 -[NSWindow _setWindowNumber:] + 684
1 AppKit 0x00000001933050e4 _NXCreateWindow + 284
2 AppKit 0x0000000192901ae0 -[NSWindow _commonAwake] + 672
3 AppKit 0x000000019281ff00 -[NSWindow _commonInitFrame:styleMask:backing:defer:] + 972
4 AppKit 0x000000019281f798 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 796
5 AppKit 0x000000019281f470 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 48
6 flutter_desktop_darwin_unittests 0x0000000100001e3c _ZN7flutter7testing89AccessibilityBridgeMacTest_SendsAccessibilityCreateNotificationToWindowOfFlutterView_Test8TestBodyEv + 328
```
See: http://www.openradar.me/FB13291861
Issue: https://github.com/flutter/flutter/issues/104789
Issue: https://github.com/flutter/flutter/issues/127441
Issue: https://github.com/flutter/flutter/issues/124840
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This PR relands https://github.com/flutter/engine/pull/44473.
The previous PR was immediately reverted after merging because we found that the PR could cause illegal renders to be skipped on debug builds but crash the app on release builds. This PR makes the `Animator::Render` skip illegal renders as well. This should not be the final shape of this feature, and thus a TODO is added.
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This reverts commit 670b5bd8e714a1fd2bd137aa2c5b36760b8ce89b.
With https://github.com/flutter/engine/pull/46911, the Platform.script API will return a URI for the compiled temporary wrapper script generated by "flutter test". This will cause issues for tests that expect Platform.script to reflect the directory where "flutter test" was launched.
This addresses the problem in https://github.com/flutter/flutter/issues/12847 which changed slightly over time.
Today, `Platform.script` does not give an empty `file` URI, it gives something like `file://path/to/package/main.dart` _regardless of how the file is actually named_.
After this change, it will give the absolute path to the file being run under test.
So before this change, the new test would have a URI like
```
file:///Users/dnfield/src/flutter/engine/src/main.dart
```
And now it has
```
file:///Users/dnfield/src/flutter/engine/src/out/host_debug_unopt_arm64/gen/platform_test.dart.dill
```
This is going to be helpful in generating relative paths from the test file.
The Impeller ContextVK contains a ConcurrentMessageLoop whose threads may invoke Dart timeline APIs. The Dart APIs will create a thread-local object that will be deleted during thread shutdown. Therefore, these threads should not outlive the engine/Shell and Dart VM.
Previously, RunTester held the ImpellerVulkanContextHolder on the stack, and its reference to the ContextVK would be dropped while exiting the function after the Shell is destructed.
This PR moves the contents of the tester's ImpellerVulkanContextHolder out of the instance on the stack and into a lambda owned by the Shell.
It also reenables the flutter_tester Impeller tests in the run_tests script.
When objcopy is used to embed data into a linkable object file, that
object file will only have default bits set in its header for ABI etc..
If the linker doesn't cooperate by ignoring ABI mismatches on object
files without code, then linking will fail. This PR stops using objcopy
to create an object file that embeds icudtl.dat into the Android
embedder, and instead uses the `bin_to_assembly.py` script that we're
already using for Dart VM snapshot data.
Context in https://github.com/llvm/llvm-project/issues/68915
Instead, this PR copies the frontend server from the Dart SDK (whether prebuilt or not) into the location expected by internal engine tests and artifact construction. This PR also consolidates the three GN templates that invoked the frontend server down to one.
Framework presubs: https://github.com/flutter/flutter/pull/135836
Related: https://github.com/flutter/flutter/issues/60007
This patch does the following:
- Updates `flutter_tester` to set up an Impeller rendering context and surface if `--enable-impeller` is set to true, using the Vulkan backend with Swiftshader.
- Updates `run_tests.py` to run all tests except the smoke test (that one really has no rendering impact whatsoever) with and without `--enable-impeller`.
- Updates a few tests to work that were trivial:
- A couple tests needed updated goldens for very minor rendering differences. Filed https://github.com/flutter/flutter/issues/135684 to track using Skia gold for this instead.
- Disabled SKP screenshotting if Impeller is enabled, and updated the test checking that to verify an error is thrown if an SKP is requested.
- The Dart GPU based test now asserts that the gpu context is available if Impeller is enabled, and does not deadlock if run in a single threaded mode.
- We were missing some trace events around `Canvas::SaveLayer` for Impeller as compared to Skia.
- A couple other tests had strict checks about exception messages that are slightly different between Skia and Impeller.
- I've filed bugs for other tests that may require a little more work, and skipped them for now. For FragmentProgram on Vulkan I reused an existing bug.
This is part of my attempt to address https://github.com/flutter/flutter/issues/135693, although @chinmaygarde and I had slightly different ideas about how to do this.
The goals here are:
- Run the Dart unit tests we already have with Impeller enabled.
- Enable running more of the framework tests (including gold tests) with Impeller enabled.
- Run all of these tests via public `dart:ui` API rather than mucking around in C++ internals in the engine.
Switch the impeller testing in rendertests to use TextFrame objects rather than TextBlob objects when drawing to an Impeller-managed destination.
Previously the DrawText tests would generate over 200 errors because none of the attribute variants would render anything at all. After this fix the number of failures in the DrawText calls is down to 1 which points out that wrapping a DrawTextFrame call in a nothing saveLayer somehow affects the glyph positioning (in a not very appealing manner).
This PR creates a new test type for `run_tests.py` called `dart-host`.
The tests remaining under the `dart` type are tests that run in
`flutter_tester`. The `dart-host` tests run in the Dart CLI. This allows
`run_tests.py` to be simplified a bit, and while doing this I spotted a
couple of mistakes that were causing some tests not to run.
This PR also makes a couple of small style fixes to the build config
json files, converting tabs to spaces, and moving some "script" fields
before the "parameters" fields.
Due to https://github.com/flutter/flutter/issues/127500 , we can get in a state where enable-impeller is true but we're using Skia. We need to either fall back completely to Skia, make this configuration fatal, or remote the check
----------------
Conversion of SkTextBlobs to impeller::TextFrame objects is one of the most expensive operations in display list dispatching. While the rest of the engine and framework makes a reasonable attempt to cache the SkTextBlobs generated during paragraph construction, the design of the dl dispatcher means that these the Impeller backend will always reconstruct all text frames on each frame - even if the display list/picture that contained those text frames was unchanged.
Removing this overhead is one of the goals of https://github.com/flutter/engine/pull/45386 , however this patch is also fairly risky and will be difficult to land. As a more incremental solution, we can instead construct the impeller::TextFrame objects when performing paragraph painting and record them in the display list. This both moves the text frame construction to the UI thread and allows the framework/engine to cache unchanged text frames.
This also does not conflict with the dl_aiks_canvas patch directly, and is fine to land before or after it does. (though I'd argue we should land this first).
To compare the current performance levels, I ran the complex_layout_scroll perf test, since this is fairly text filled. On a Pixel 6 pro. Across several runs this is a fairly consistent ~1ms raster time improvement.
Fixes https://github.com/flutter/flutter/issues/133204
Closes https://github.com/flutter/flutter/issues/135057.
This is a fair bit more involved than previous changes, just due to the sheer number of implicit conversions.
Highlights:
- Made `public uint32_t argb` `private uint32_t argb_`, and added `argb()` instead.
- Added `ToSk(DlColor)` instead of using implicit conversions.
There were a bunch of places where I had to make a judgement call (particularly in tests) to keep the code a bit "messy", i.e. `DlColor(SK_RED)`, just to make the diff as small as possible and to prevent silly copy and paste bugs. I'd be open to filing a follow-up issue to reduce unnecessary wrapping.
TwoPlatformViewsWithOtherBackDropFilterTests is failing on macOS 13 with the same simulator version used in macOS 12. The image diff looks identical and slightly above the threshold. This PR adjusts the threshold for this test temporarily so our CI can run on both macOS 13 and 12. This change can be reverted when we move all our CI to macOS 13
Fixes https://github.com/flutter/flutter/issues/134740
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
Reverts flutter/engine#45418
Some google3 tests are hitting the CHECK I added in the DlSkCanvasDispatcher::drawTextFrame, which indicates that the SkParagraph code likely thinks impeller is enabled, whereas other code might be running with Skia.
Perhaps this could happen if its software rendering? It should be a fatal error on startup so we can track this down.
This PR changes run_tests.py to use the python logging library to report
results instead of direct prints or writes to stdout/stderr. This change
simplifies adding a `--quiet` flag that causes the script to only
generate output if a log is emitted at WARNING or above.
Overall this is a bit of progress toward landing something like
https://github.com/flutter/engine/pull/45595
flutter/flutter/#134220
Command required to update lock files required not using the custom task and required matching the version of gradle in ci and passing a custom version of java.
`~/.gradle/wrapper/dists/gradle-7.5.1-bin/7jzzequgds1hbszbhq3npc5ng/gradle-7.5.1/bin/gradle -D=org.gradle.java.home=/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home app:dependencies --write-locks` for app:dependencies and dependencies.
Conversion of SkTextBlobs to impeller::TextFrame objects is one of the most expensive operations in display list dispatching. While the rest of the engine and framework makes a reasonable attempt to cache the SkTextBlobs generated during paragraph construction, the design of the dl dispatcher means that these the Impeller backend will always reconstruct all text frames on each frame - even if the display list/picture that contained those text frames was unchanged.
Removing this overhead is one of the goals of https://github.com/flutter/engine/pull/45386 , however this patch is also fairly risky and will be difficult to land. As a more incremental solution, we can instead construct the impeller::TextFrame objects when performing paragraph painting and record them in the display list. This both moves the text frame construction to the UI thread and allows the framework/engine to cache unchanged text frames.
This also does not conflict with the dl_aiks_canvas patch directly, and is fine to land before or after it does. (though I'd argue we should land this first).
To compare the current performance levels, I ran the complex_layout_scroll perf test, since this is fairly text filled. On a Pixel 6 pro. Across several runs this is a fairly consistent ~1ms raster time improvement.
### Skia
```
"average_frame_build_time_millis": 1.497333333333333,
"90th_percentile_frame_build_time_millis": 2.038,
"99th_percentile_frame_build_time_millis": 17.686,
"worst_frame_build_time_millis": 23.095,
"missed_frame_build_budget_count": 3,
"average_frame_rasterizer_time_millis": 5.5078589743589745,
"stddev_frame_rasterizer_time_millis": 2.226343414420338,
"90th_percentile_frame_rasterizer_time_millis": 7.481,
"99th_percentile_frame_rasterizer_time_millis": 19.11,
"worst_frame_rasterizer_time_millis": 79.799,
"missed_frame_rasterizer_budget_count": 7,
"frame_count": 234,
"frame_rasterizer_count": 234,
"new_gen_gc_count": 10,
"old_gen_gc_count": 2,
```
### Impeller (ToT)
```
"average_frame_build_time_millis": 1.431575000000001,
"90th_percentile_frame_build_time_millis": 2.196,
"99th_percentile_frame_build_time_millis": 14.486,
"worst_frame_build_time_millis": 23.728,
"missed_frame_build_budget_count": 2,
"average_frame_rasterizer_time_millis": 6.536087499999999,
"stddev_frame_rasterizer_time_millis": 1.9902712500000004,
"90th_percentile_frame_rasterizer_time_millis": 9.705,
"99th_percentile_frame_rasterizer_time_millis": 14.727,
"worst_frame_rasterizer_time_millis": 17.838,
"missed_frame_rasterizer_budget_count": 1,
"frame_count": 240,
"frame_rasterizer_count": 240,
"new_gen_gc_count": 10,
"old_gen_gc_count": 2,
```
### Impeller (Patched)
```
"average_frame_build_time_millis": 1.4500167364016743,
"90th_percentile_frame_build_time_millis": 2.478,
"99th_percentile_frame_build_time_millis": 14.883,
"worst_frame_build_time_millis": 18.782,
"missed_frame_build_budget_count": 1,
"average_frame_rasterizer_time_millis": 5.023033333333336,
"stddev_frame_rasterizer_time_millis": 1.6445388888888894,
"90th_percentile_frame_rasterizer_time_millis": 7.814,
"99th_percentile_frame_rasterizer_time_millis": 13.497,
"worst_frame_rasterizer_time_millis": 15.008,
"missed_frame_rasterizer_budget_count": 0,
"frame_count": 239,
"frame_rasterizer_count": 240,
"new_gen_gc_count": 8,
"old_gen_gc_count": 0,
```
