Relands https://github.com/flutter/engine/pull/45131
Fixes https://github.com/flutter/flutter/issues/132416
Differences from last time:
- Some minor merge conflict fixes
- Use the RTree to get the bounds instead of recalculating the bounds
- Make the iOS platform view controller implementation use the impeller-aware slices instead of the display list ones. This has been fixed for Android and the desktop embedding, but I missed iOS. The unit tests weren't actually running before I branched for my PR, @zanderso fixed them up separately and this resulted in catching the failures on post submit last time.
Reverts flutter/engine#45131
This is failing the Impeller variants of the unobstructed platform views tests:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/132249/overview
```
Failing tests:
-[UnobstructedPlatformViewTests testPlatformViewsMaxOverlays]
-[UnobstructedPlatformViewTests testOneOverlay]
-[UnobstructedPlatformViewTests testOneOverlayPartialIntersection]
-[UnobstructedPlatformViewTests testTwoIntersectingOverlays]
-[UnobstructedPlatformViewTests testOneOverlayAndTwoIntersectingOverlays]
** TEST FAILED **
```
We've retried it a few times so I suspect this isn't a flake.
This is a reland of https://github.com/flutter/engine/pull/44248
Fixes https://github.com/flutter/flutter/issues/132416
Changes from last time:
- The `drawPoints` benchmark was failing to render anything meaningful because of an error in `AiksLayer` that resulted in an infinitely sized bounding rectangle poisoning the bounds with `NaN` (this happens, for example, with a `drawPaint` call, which that benchmark happens to use). Added a test covering this and filed https://github.com/flutter/flutter/issues/132770 to explore ways to avoid this in the future.
- There was a bug in `DlAiksCanvas::SaveLayer` where a `nullptr` `paint` but non-`nullptr` `backdrop` was failing to actually save the layer. This resulted in incorrect rendering.
- There was a bug in `impeller::Canvas::DrawPicture` that resulted in incorrect stencil depth counting. That was fixed separately by @bdero, but was the cause of incorrect rendering in some Wonderous screens.
- I've added a simple implementation for `AiksLayer::IsReplacing`. It does not currently compare as deeply as the `DisplayListLayer` version potentially does, but it is good enough to avoid the regression noted in https://github.com/flutter/flutter/issues/132071. That regression was on a benchmark that greatly benefits from partial repaint. With the new implementation, it still gains partial repaint where it previously did not. There is more work that can be done here, filed https://github.com/flutter/flutter/issues/133361 to track that work.
I merged but did not fully integrate the `DisplayListBuilder`/`CanvasToReceiver` work that @flar has done. I have a local experiment with that, but would prefer to see this land and run through the device lab so we get some better comparison numbers for which one performs better.
Resolves https://github.com/flutter/flutter/issues/130947.
- Allow for setting the text renderer (`TextRenderContext`) when the
`AiksContext` is built. Previously, the text renderer was selected at
build time through linking a `TextRenderContext::Create` symbol.
- Support using Impeller without a text backend. Throw a clear error
when trying to render text if no text backend is present.
- Don't track the Impeller context in `TextRenderContext`. Just let the
renderer supply its context when generating atlases.
- Allow for overriding the `TextRenderContext` for individual Aiks
playground tests/goldens.
Fixes https://github.com/flutter/flutter/issues/130141
The primary goal of this patch is to move dispatching of `dart:ui` `Canvas` commands to the UI thread.
Before this patch, the architecture is something like:
## UI Thread
- `dart:ui` talks to `DisplayListBuilder`, a `DlCanvas` implementation.
- `DisplayListBuilder` does some clip/bounds tracking and creates a `DisplayList` object that is held by `dart:ui`'s `Picture` objects.
- `DisplayList`s are added to `DisplayListLayer`s in `flow`.
## Raster Thread
- `flow` flattens the various operations into a single `DisplayList` via another `DisplayListBuilder`.
- A `DlOpReceiver`implementation converts that `DisplayList` into an `Aiks` `Canvas`/`Picture`.
After this patch, the architecture instead looks like:
## UI Thread
- No change for Skia.
- If Impeller, use a new `DlCanvasImplementation` that talks to `Aiks`'s `Canvas`.
- If Impeller, `dart:ui` Picture's now hold an `Aiks` `Picture`, which get shared into `AiksLayer`s in `flow`.
## Raster thread
- No change for Skia, but some light refactoring for places that assumed a `DisplayListBuilder` where they really just needed a `DlCanvas`.
- The `Aiks` `Picture`s are combined using new API on `DlCanvas` and still backed by `Aiks`.
These changes show significant improvement on raster times on Android and only very small regressions on UI times in local testing, see https://gist.github.com/dnfield/26528090194c9f5abdbac13cdcbf4f79 for old gallery transition perf numbers.
Many of the other changes in this patch are related to the following:
- Making `DlRTree` usable for Impeller.
- It would be nice to have a version of DlRTree that speaks `impeller::Rect`.
- Creating the requisite classes to support `EmbeddedViews` so that Desktop works.
This patch does not remove the `impeller::DlDispatcher`, which now would only be used in tests.
In https://github.com/flutter/engine/pull/41059 the Android context and surface were changed to share one impeller::ContextVK across all instances of AndroidSurfaceVulkanImpeller. However, the ContextVK holds a reference to the current swapchain. If an app uses multiple AndroidSurfaceVulkanImpeller instances (e.g. for platform views), then one surface may overwrite the ContextVK's swapchain while another surface is trying to render.
This patch allows each surface to get its own impeller::Context instance holding a swapchain tied to that surface. The per-surface contexts will delegate most operations to the shared parent ContextVK.
Actually works now, though there is some issue with the default fbo stencil so I've filled https://github.com/flutter/flutter/issues/130048
Other issues:
* ~~Rendering looks wrong~~
* ~~Resizing window hangs~~
* ~~Reactor isn't set up correctly and all blit passes are currently failing.~~
* ~~Needs to handle falling back to sample count of 1 like we do on Android~~.
Switching the calls to dispatch into an Impeller Dispatcher to use a cull rect to enable pre-culling of the out-of-bounds ops.
This change showed an improvement of around 2x on the rendering performance of the non-intersecting platform view benchmark, but that was measured without the recent changes to the destructive blend modes in Impeller renderer.
Implements partial repaint for Impeller.
Fixes https://github.com/flutter/flutter/issues/124526
The new code that manages the damage regions is more or less a copy paste from the existing Skia implementation. Compared to Skia, there are a few differences:
Normally Impeller wants to use the drawable as the resolve texture for the root MSAA pass. Unfortunately this will unconditonally clear that texture. Thus to do a partial repaint, we have to allocate a separate texture to resolve to and then blit into the drawable.
The blit seems to take about 500ns for a full screen on an iPhone 13. That implies that partial repaint is likely not worth doing if the screen is significantly changed. Thus I've added code in compositor_context.cc that computes the percentage of width or height that is part of the dirty rect. Above a threshold of (abitrarily chosen) 70%, we just render as normal. This should mean there is only a very minor hit from performing the diff on screens that are highly changed.
The other special case, is that sometimes we get damage rects that are empty - that is the drawable is already completely up to date with what we want to render. IN that case I shortcircuit all of the impeller code and just present immediately. I previously tried returning without a present but this resulted in Xcode reporting dropped frames. One caveat here is that if you use the XCode frame debugger and attempt to capture a frame where we early present, then it will claim it couldn't capture any command buffers (because we didn't create any).
To facilitate all of this, I added some additonal plumbing so that the impeller surface can get the clip rect from the submit info. Additionally, rather than using a clip rect impeller will translate and then shrink the root surface texture. This reduces memory usage compared to just clippling.
No change in functionality. I've just renamed the TU's to replace `display_list_` to `dl_`. I am sure there are chances to make the naming better. For instance, the op receiver still being named a dispatcher.
Kaushik already did most of the great work. This patch gets us to a point where
all the playgrounds tests pass and the common performance optimizations are in
place. All known non-compute Impeller features should be implemented. There are
iOS only performance optimizations that haven’t been opted into because these
are behind define guards. These need to be generalized. Without that, the
performance of this backend will lag behind that of the Metal backend but only
because it is not an apples to apples comparison.
The general rubric was to keep the code and concepts as similar to the Metal
backend as possible.
The list of updates, all to the Vulkan Impeller backend:
* MSAA is wired up.
* Depth and stencil attachments and pipeline states are wired up.
* Got rid of Vulkan specific hacks in the inline pass context that were
preventing clips from working.
* Storage modes for both device buffer and texture allocation are respected.
This includes optimal usage of tile memory for device transient attachments.
* Host coherent memory for textures is no longer a requirement with explicit
mapping management and write flushes.
* Texture uploads should be optimized for the UMA case without needing a staging
buffer. That entire pipeline has been reworked.
* Textures track their current layout and ensure they get to the right layout
based on usage without redundant transitions.
* Cube textures are now supported.
* Mipmapping has been reworked to correct image layout errors. With the new
texture layout transition management, blit passes should be a whole lot easier
to read and reason about.
* Allocator allocations are named using Vulkan debug utilities and the allocator
(VMA). Comes in handy when chasing leaks. All of which are chased down.
* Left some handy utilities in there to debug resource leaks after context
shutdown. These are validation errors.
* Debug groups are pushed around render pass command encoding as well as the
entire pass contents. This mimics the behavior of the Metal backend and it
should be easy to map traces using both backends in Xcode and RenderDoc.
* Command buffer submission allows for tracking the life cycles of all resources
referenced in the command stream and ensuring that the objects stay alive till
past the submission of the command buffer. All use-after-free issues due to
this class of issue have been chased down.
* Command pools are now context global instead of being created per pass.
* Descriptor pool are now context global instead of being created per pass.
Individual descriptor types are now reference counted and returned to the pool
when not needed. The pool is still fixed size though (and hence relatively
large).
* All instances of global waitIdle on the device are removed during normal
operation. The only times a waitIdle may happen is during swapchain recreation
and context destruction.
* Swapchain adapt to them going out of date with the underlying surface and
seamlessly resize as necessary on the next drawable acquisition.
* Playgrounds are resizable.
* Pipeline front face and cull modes are respected.
* Clears for all attachments are respected.
* Maximum textures sizes supported on the device are respected instead of
hardcoding the minimum Vulkan requirement.
Fixes https://github.com/flutter/flutter/issues/112388
Fixes https://github.com/flutter/flutter/issues/112648
Fixes https://github.com/flutter/flutter/issues/112647 and a few other issues...
* Implemented wide gamut images for iOS
Moved the surface to an extended range color format.
* wrong gamma but default pixel format set to bgra10_xr
* BGR10_XR add
* format
* updated todos
* updated todo with information about pixel formats
* switched logic for determining if we have a wide gamut image
* cleaned up gamut math to match style and linked source
* made the color attachment pixel format match the surface
* updated vulkan format switch
* removed comment
* added enable disable switch
* moved default to bgr10 for now since there is a bug where someone is still reading this, msaa?
* fixed the decoder settings to make sure we don't lose wide gamut colors
* fixed stored srgb gamut variable
* fixed false lint
* updated test
* added ability to grab the surface data for tests
* made the screenshot utility return the format
* added width and height to the platform channel payload
* fixed a couple of broken targets
* moved back the default pixel buffer format
* cleanup and add docstrings
* made the surfacedata feature only available in debug builds
* added decoding unit test
* fixed objc tests
* turned off by default
* bdero feedback1
* bdero2
* bdero3
* fixed merge issue
* removed using std::shared_ptr
This ensures FlutterMetalTexture.destruction_callback gets called.
FlutterRendererConfig.get_next_drawable_callback holds a callback used by the embedder API to request a drawable; in the case of Metal, this drawable is a FlutterMetalTexture.
FlutterMetalTexture.destruction_callback should be called when it's safe to release resources associated with the FlutterMetalTexture. This callback is not currently invoked for textures returned via FlutterRendererConfig.get_next_drawable_callback; instead we unpack the returned struct and pass it on.
In the compositor codepath, we do create an SkSurface that triggers the destruction callback, here:
bbdb5d6a3e/shell/platform/embedder/embedder.cc (L868-L881)
Issue: https://github.com/flutter/flutter/issues/116381
If the following is added to Info.plist of the application,
it will disable partial repaint:
```
<key>FLTDisablePartialRepaint</key>
<true/>
```
The primary intended usecase for this is to enable applications to see if disabling partial repaint resolves:
https://github.com/flutter/flutter/issues/100522.
Roll Swiftshader, ANGLE, and Vulkan deps.
Updates license bot for new locations of licenses and new files
in roll.
Converts unit test harnesses to use SwANGLE, which is the
supported way of using SwiftShader as a software backend for GLES.
Updates goldens due to swiftshader changes.
Fixes up include paths to avoid directly including third_party
code by that name, which is unlikely to work in other build
environments.
Includes upstream patches to ANGLE and SwiftShader to build in our
windows environments, as well as patches to Skia to support
defining the location of VulkanMemoryAllocator and a patch to ANGLE
to support opting into a newer version of VMA.
Merges most (but not all) of the impeller .clang-tidy rules into the
main .clang-tidy config. Merges:
readability-identifier-naming.PrivateMemberSuffix (_)
readability-identifier-naming.EnumConstantPrefix (k)
modernize-use-default-member-init.UseAssignment
Does not merge:
readability-identifier-naming.PublicMethodCase (CamelCase)
readability-identifier-naming.PrivateMethodCase (CamelCase)
These last two are not merged due to the non-trivial number of existing
field accessors that use field_name() methods to directly return
field_name_. While these are permitted by the C++ style guide, we may
want to move to a single, simple rule and name everything in CamelCase.
These can be enabled in a followup patch.
No new tests added, since this change is style-only.
Towards implementing the FragmentProgram API in Impeller.
Specifies an Impeller specific format for data the renderer can use to create
pipelines with user supplied shader stages at runtime.
The data is in the form of a flatbuffer with a known schema.
This patch implements the wire format, creating and loading the program
payloads, and creating pipeline state objects using these payloads.
If the user supplied SPIRV intended for the older API, the loader will reject
this invalid payload. This is probably not going to be too much of an issue
because the FragmentProgram API will probably be modified to only allow buffers
loaded from asset managers. But still, in the meantime, I am using the old API
to pass these new buffers.
Fixes https://github.com/flutter/flutter/issues/104750
Fixes https://github.com/flutter/flutter/issues/105542
Towards resolving https://github.com/flutter/flutter/issues/102853
