Without this, the EmbedderExternalViewEmbedder's call to SurfaceFrame::Submit will render an empty display list that will overwrite the output rendered by the external view embedder's layers.
See https://github.com/flutter/flutter/issues/143387
Fixes https://github.com/flutter/flutter/issues/141351 (speculatively - I have not directly reproduced this in an application, but without this change the added test crashes with a segfault in the submit callback).
If the rasterizer gets torn down, the surface gets released and the submit callback may fire on a collected object. Capturing `this` isn't safe. I'm not quite sure how that could happen from the linked stack trace though, since the draw call and the teardown call should be happening on the raster thread, and if the surface was reset then the draw call should've failed earlier...
The added test causes a segfault without the change.
Reland of https://github.com/flutter/engine/pull/49505
---
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
Reverts flutter/engine#49505
Initiated by: jonahwilliams
This change reverts the following previous change:
Original Description:
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
part of https://github.com/flutter/flutter/issues/140804
We can't use the existing host buffer abstraction as that requires us to collect all allocations up front. By itself, this isn't sufficient for #140804 , because we'll need a way to mark ranges as dirty and/or flush if we don't have host coherent memory. But by itself this change should be beneficial as we'll create fewer device buffers and should do less allocation in general.
The size of the device buffers is 1024 Kb, somewhat arbitrarily chosen.
This is more-or-less a revert of https://github.com/flutter/engine/pull/14011
This code never ended up being used outside of tests, and it's not how we handle asset loading at this point anyway.
I was hopeful we could kill off all runtime dependencies on Dart in `FML` when looking at this, but it looks like trace_event.h still wants to import dart_api_tools.h for some Dart enum types. This may or may not matter if we ever want to build FML for web/wasm. /cc @eyebrowsoffire. If we really need to do that, we can refactor the trace event stuff so that it has a web and Dart implementation that's selected at build time.
The new `BUILD.gn` files in the Engine tree can't go under
`build/secondary` because Skia still has its own, and they'd be selected
first. So, this PR puts the new `BUILD.gn` files under `flutter/skia`.
This patch does the following:
- Updates `flutter_tester` to set up an Impeller rendering context and surface if `--enable-impeller` is set to true, using the Vulkan backend with Swiftshader.
- Updates `run_tests.py` to run all tests except the smoke test (that one really has no rendering impact whatsoever) with and without `--enable-impeller`.
- Updates a few tests to work that were trivial:
- A couple tests needed updated goldens for very minor rendering differences. Filed https://github.com/flutter/flutter/issues/135684 to track using Skia gold for this instead.
- Disabled SKP screenshotting if Impeller is enabled, and updated the test checking that to verify an error is thrown if an SKP is requested.
- The Dart GPU based test now asserts that the gpu context is available if Impeller is enabled, and does not deadlock if run in a single threaded mode.
- We were missing some trace events around `Canvas::SaveLayer` for Impeller as compared to Skia.
- A couple other tests had strict checks about exception messages that are slightly different between Skia and Impeller.
- I've filed bugs for other tests that may require a little more work, and skipped them for now. For FragmentProgram on Vulkan I reused an existing bug.
This is part of my attempt to address https://github.com/flutter/flutter/issues/135693, although @chinmaygarde and I had slightly different ideas about how to do this.
The goals here are:
- Run the Dart unit tests we already have with Impeller enabled.
- Enable running more of the framework tests (including gold tests) with Impeller enabled.
- Run all of these tests via public `dart:ui` API rather than mucking around in C++ internals in the engine.
# Description
This PR fixes the `gl_populate_existing_damage` in embedder.cc, which currently returns an empty rectangle when a full repaint is needed. This leads to the `frame_damage` being considered empty in rasterizer.cc, causing incorrect partial repaint within Flutter when `gl_populate_existing_damage` is not provided by the embedder.
# Related Issue
https://github.com/flutter/flutter/issues/119601
# Tests
Add a new test. Also fixes damage calculation related tests in EmbedderTest by
* Use a new Dart embedder fixture entry point `render_gradient_retained` which retains the old layer to make sure layer tree diff happens.
* Add `latch.Wait()` after `SetGLPresentCallback` to make sure assertions have been executed.
* Make `existing_damage_rects` static since it should be valid after `populate_existing_damage` returns.
In https://skia-review.googlesource.com/c/skia/+/742797 Skia refactored
GrBackend* to not require #ifdefs. This changes
callsites in Flutter to use static functions instead of methods that
were conditionally compiled on those classes.
There should be no functional change.
Relands https://github.com/flutter/engine/pull/45131
Fixes https://github.com/flutter/flutter/issues/132416
Differences from last time:
- Some minor merge conflict fixes
- Use the RTree to get the bounds instead of recalculating the bounds
- Make the iOS platform view controller implementation use the impeller-aware slices instead of the display list ones. This has been fixed for Android and the desktop embedding, but I missed iOS. The unit tests weren't actually running before I branched for my PR, @zanderso fixed them up separately and this resulted in catching the failures on post submit last time.
Reverts flutter/engine#45131
This is failing the Impeller variants of the unobstructed platform views tests:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/132249/overview
```
Failing tests:
-[UnobstructedPlatformViewTests testPlatformViewsMaxOverlays]
-[UnobstructedPlatformViewTests testOneOverlay]
-[UnobstructedPlatformViewTests testOneOverlayPartialIntersection]
-[UnobstructedPlatformViewTests testTwoIntersectingOverlays]
-[UnobstructedPlatformViewTests testOneOverlayAndTwoIntersectingOverlays]
** TEST FAILED **
```
We've retried it a few times so I suspect this isn't a flake.
This is a reland of https://github.com/flutter/engine/pull/44248
Fixes https://github.com/flutter/flutter/issues/132416
Changes from last time:
- The `drawPoints` benchmark was failing to render anything meaningful because of an error in `AiksLayer` that resulted in an infinitely sized bounding rectangle poisoning the bounds with `NaN` (this happens, for example, with a `drawPaint` call, which that benchmark happens to use). Added a test covering this and filed https://github.com/flutter/flutter/issues/132770 to explore ways to avoid this in the future.
- There was a bug in `DlAiksCanvas::SaveLayer` where a `nullptr` `paint` but non-`nullptr` `backdrop` was failing to actually save the layer. This resulted in incorrect rendering.
- There was a bug in `impeller::Canvas::DrawPicture` that resulted in incorrect stencil depth counting. That was fixed separately by @bdero, but was the cause of incorrect rendering in some Wonderous screens.
- I've added a simple implementation for `AiksLayer::IsReplacing`. It does not currently compare as deeply as the `DisplayListLayer` version potentially does, but it is good enough to avoid the regression noted in https://github.com/flutter/flutter/issues/132071. That regression was on a benchmark that greatly benefits from partial repaint. With the new implementation, it still gains partial repaint where it previously did not. There is more work that can be done here, filed https://github.com/flutter/flutter/issues/133361 to track that work.
I merged but did not fully integrate the `DisplayListBuilder`/`CanvasToReceiver` work that @flar has done. I have a local experiment with that, but would prefer to see this land and run through the device lab so we get some better comparison numbers for which one performs better.
Resolves https://github.com/flutter/flutter/issues/130947.
- Allow for setting the text renderer (`TextRenderContext`) when the
`AiksContext` is built. Previously, the text renderer was selected at
build time through linking a `TextRenderContext::Create` symbol.
- Support using Impeller without a text backend. Throw a clear error
when trying to render text if no text backend is present.
- Don't track the Impeller context in `TextRenderContext`. Just let the
renderer supply its context when generating atlases.
- Allow for overriding the `TextRenderContext` for individual Aiks
playground tests/goldens.
Fixes https://github.com/flutter/flutter/issues/130141
The primary goal of this patch is to move dispatching of `dart:ui` `Canvas` commands to the UI thread.
Before this patch, the architecture is something like:
## UI Thread
- `dart:ui` talks to `DisplayListBuilder`, a `DlCanvas` implementation.
- `DisplayListBuilder` does some clip/bounds tracking and creates a `DisplayList` object that is held by `dart:ui`'s `Picture` objects.
- `DisplayList`s are added to `DisplayListLayer`s in `flow`.
## Raster Thread
- `flow` flattens the various operations into a single `DisplayList` via another `DisplayListBuilder`.
- A `DlOpReceiver`implementation converts that `DisplayList` into an `Aiks` `Canvas`/`Picture`.
After this patch, the architecture instead looks like:
## UI Thread
- No change for Skia.
- If Impeller, use a new `DlCanvasImplementation` that talks to `Aiks`'s `Canvas`.
- If Impeller, `dart:ui` Picture's now hold an `Aiks` `Picture`, which get shared into `AiksLayer`s in `flow`.
## Raster thread
- No change for Skia, but some light refactoring for places that assumed a `DisplayListBuilder` where they really just needed a `DlCanvas`.
- The `Aiks` `Picture`s are combined using new API on `DlCanvas` and still backed by `Aiks`.
These changes show significant improvement on raster times on Android and only very small regressions on UI times in local testing, see https://gist.github.com/dnfield/26528090194c9f5abdbac13cdcbf4f79 for old gallery transition perf numbers.
Many of the other changes in this patch are related to the following:
- Making `DlRTree` usable for Impeller.
- It would be nice to have a version of DlRTree that speaks `impeller::Rect`.
- Creating the requisite classes to support `EmbeddedViews` so that Desktop works.
This patch does not remove the `impeller::DlDispatcher`, which now would only be used in tests.
In https://github.com/flutter/engine/pull/41059 the Android context and surface were changed to share one impeller::ContextVK across all instances of AndroidSurfaceVulkanImpeller. However, the ContextVK holds a reference to the current swapchain. If an app uses multiple AndroidSurfaceVulkanImpeller instances (e.g. for platform views), then one surface may overwrite the ContextVK's swapchain while another surface is trying to render.
This patch allows each surface to get its own impeller::Context instance holding a swapchain tied to that surface. The per-surface contexts will delegate most operations to the shared parent ContextVK.
Actually works now, though there is some issue with the default fbo stencil so I've filled https://github.com/flutter/flutter/issues/130048
Other issues:
* ~~Rendering looks wrong~~
* ~~Resizing window hangs~~
* ~~Reactor isn't set up correctly and all blit passes are currently failing.~~
* ~~Needs to handle falling back to sample count of 1 like we do on Android~~.
Switching the calls to dispatch into an Impeller Dispatcher to use a cull rect to enable pre-culling of the out-of-bounds ops.
This change showed an improvement of around 2x on the rendering performance of the non-intersecting platform view benchmark, but that was measured without the recent changes to the destructive blend modes in Impeller renderer.
Implements partial repaint for Impeller.
Fixes https://github.com/flutter/flutter/issues/124526
The new code that manages the damage regions is more or less a copy paste from the existing Skia implementation. Compared to Skia, there are a few differences:
Normally Impeller wants to use the drawable as the resolve texture for the root MSAA pass. Unfortunately this will unconditonally clear that texture. Thus to do a partial repaint, we have to allocate a separate texture to resolve to and then blit into the drawable.
The blit seems to take about 500ns for a full screen on an iPhone 13. That implies that partial repaint is likely not worth doing if the screen is significantly changed. Thus I've added code in compositor_context.cc that computes the percentage of width or height that is part of the dirty rect. Above a threshold of (abitrarily chosen) 70%, we just render as normal. This should mean there is only a very minor hit from performing the diff on screens that are highly changed.
The other special case, is that sometimes we get damage rects that are empty - that is the drawable is already completely up to date with what we want to render. IN that case I shortcircuit all of the impeller code and just present immediately. I previously tried returning without a present but this resulted in Xcode reporting dropped frames. One caveat here is that if you use the XCode frame debugger and attempt to capture a frame where we early present, then it will claim it couldn't capture any command buffers (because we didn't create any).
To facilitate all of this, I added some additonal plumbing so that the impeller surface can get the clip rect from the submit info. Additionally, rather than using a clip rect impeller will translate and then shrink the root surface texture. This reduces memory usage compared to just clippling.
No change in functionality. I've just renamed the TU's to replace `display_list_` to `dl_`. I am sure there are chances to make the naming better. For instance, the op receiver still being named a dispatcher.
Kaushik already did most of the great work. This patch gets us to a point where
all the playgrounds tests pass and the common performance optimizations are in
place. All known non-compute Impeller features should be implemented. There are
iOS only performance optimizations that haven’t been opted into because these
are behind define guards. These need to be generalized. Without that, the
performance of this backend will lag behind that of the Metal backend but only
because it is not an apples to apples comparison.
The general rubric was to keep the code and concepts as similar to the Metal
backend as possible.
The list of updates, all to the Vulkan Impeller backend:
* MSAA is wired up.
* Depth and stencil attachments and pipeline states are wired up.
* Got rid of Vulkan specific hacks in the inline pass context that were
preventing clips from working.
* Storage modes for both device buffer and texture allocation are respected.
This includes optimal usage of tile memory for device transient attachments.
* Host coherent memory for textures is no longer a requirement with explicit
mapping management and write flushes.
* Texture uploads should be optimized for the UMA case without needing a staging
buffer. That entire pipeline has been reworked.
* Textures track their current layout and ensure they get to the right layout
based on usage without redundant transitions.
* Cube textures are now supported.
* Mipmapping has been reworked to correct image layout errors. With the new
texture layout transition management, blit passes should be a whole lot easier
to read and reason about.
* Allocator allocations are named using Vulkan debug utilities and the allocator
(VMA). Comes in handy when chasing leaks. All of which are chased down.
* Left some handy utilities in there to debug resource leaks after context
shutdown. These are validation errors.
* Debug groups are pushed around render pass command encoding as well as the
entire pass contents. This mimics the behavior of the Metal backend and it
should be easy to map traces using both backends in Xcode and RenderDoc.
* Command buffer submission allows for tracking the life cycles of all resources
referenced in the command stream and ensuring that the objects stay alive till
past the submission of the command buffer. All use-after-free issues due to
this class of issue have been chased down.
* Command pools are now context global instead of being created per pass.
* Descriptor pool are now context global instead of being created per pass.
Individual descriptor types are now reference counted and returned to the pool
when not needed. The pool is still fixed size though (and hence relatively
large).
* All instances of global waitIdle on the device are removed during normal
operation. The only times a waitIdle may happen is during swapchain recreation
and context destruction.
* Swapchain adapt to them going out of date with the underlying surface and
seamlessly resize as necessary on the next drawable acquisition.
* Playgrounds are resizable.
* Pipeline front face and cull modes are respected.
* Clears for all attachments are respected.
* Maximum textures sizes supported on the device are respected instead of
hardcoding the minimum Vulkan requirement.
Fixes https://github.com/flutter/flutter/issues/112388
Fixes https://github.com/flutter/flutter/issues/112648
Fixes https://github.com/flutter/flutter/issues/112647 and a few other issues...
* Implemented wide gamut images for iOS
Moved the surface to an extended range color format.
* wrong gamma but default pixel format set to bgra10_xr
* BGR10_XR add
* format
* updated todos
* updated todo with information about pixel formats
* switched logic for determining if we have a wide gamut image
* cleaned up gamut math to match style and linked source
* made the color attachment pixel format match the surface
* updated vulkan format switch
* removed comment
* added enable disable switch
* moved default to bgr10 for now since there is a bug where someone is still reading this, msaa?
* fixed the decoder settings to make sure we don't lose wide gamut colors
* fixed stored srgb gamut variable
* fixed false lint
* updated test
* added ability to grab the surface data for tests
* made the screenshot utility return the format
* added width and height to the platform channel payload
* fixed a couple of broken targets
* moved back the default pixel buffer format
* cleanup and add docstrings
* made the surfacedata feature only available in debug builds
* added decoding unit test
* fixed objc tests
* turned off by default
* bdero feedback1
* bdero2
* bdero3
* fixed merge issue
* removed using std::shared_ptr
